They told me to.

Sending specific traffic over a VPN connection on OSX

For a while I’ve had to go back and forth between using the “Send all traffic over VPN connection” setting and not using it. Generally I want the setting off, so that all my network traffic doesn’t get slowed down as it bounces through extra servers. However, sometimes I need to access computers in the office that aren’t on the default VPN subnet.

Recently I found this link: http://blog.liip.ch/archive/2006/01/07/changing-default-routes-on-os-x-on-vpn.html which describes how to tweak the routing settings. However, it’s a bit out of date; as of that writing, any time you used the built-in VPN setup, all traffic would go over the connection.

In any case, let’s say you have a VPN where the default subnet is 192.168.2.x but some servers are on 192.168.3.x addresses. When you connect to the VPN, requests to 192.168.2.x addresses work just fine, but trying to access the others ends up trying to send the requests over the Internet, which of course won’t work.

To fix this, create a file called /etc/ppp/ip-up and put the following in it:

#!/bin/sh
/sbin/route -n add -net 192.168.3 $IPREMOTE >> /tmp/ppp.log 2>&1

You’ll need to create the file as root, and then set it executable. Next time you connect to your VPN, the system will automatically run that file, having set the IPREMOTE environment variable to the IP address of the VPN gateway server. It will make a log of these changes in /tmp/ppp.log for future reference.

This tells your computer to always try routing 192.168.3.x addresses through that gateway. And you can stop sending *all* your network connections through the VPN. :)

When you disconnect from the VPN, it automatically removes the route, so you shouldn’t need to do any other cleanup.

2 years ago

“top” for filesystem usage…

Does your mac sometimes slow down to a crawl due to hard drive usage, and you can’t tell what’s causing it?

I whipped up a little ruby script to act like “top” for filesystem usage.

#!/usr/bin/env ruby

data_read = ''
process_times = Hash.new(0.0)

# Clear screen
print 27.chr + '[2J'

# Open a pipe to fs_usage for file calls.
data = IO.popen('fs_usage -wf filesys')

while true

  # Keep responsive to fs_usage output by reading 10 times per second.
  10.times do
    sleep 0.1

    data_read << data.read_nonblock(1048576) rescue ''
    lines = data_read.split("\n",-1)
    data_read = lines.pop.to_s

    lines.each do |line|
      elapsed,process = line[138..148],line[152..-1]
      process_times[process] += elapsed.to_f
    end
  end
  
  # Jump up to the top of the screen and print the header.
  print 27.chr + '[f'
  puts "PROCESS".ljust(30) + ' ' + 'TIME'
  puts "------------------------------ ---------"

  # Print in order of which used the most time.
  process_times.sort { |a,b| b[1] <=> a[1] }[0..19].each do |name,time|
    puts name[0,30].ljust(30) + ' ' + sprintf('%0.6f',time).rjust(9,'0')
  end

  # Clear the times for the next loop.
  process_times.each_key { |k| process_times[k] = 0.0 }

end

However, it uses “fs_usage” to get the data, which has to be run as root, so:

$ sudo ruby fs_top.rb

PROCESS                        TIME
------------------------------ ---------
ruby                           00.000542
pppd                           00.000145
firefox-bin                    00.000000
iTunes                         00.000000
update                         00.000000
cron                           00.000000
thunderbird-bin                00.000000
notifyd                        00.000000
SystemUIServer                 00.000000

Then just hit ctrl+c to quit. Enjoy.

2 years ago

I’m just sayin’…

Cheetahs Never Lose: 8
Ring of Fire: 9

http://www.sandlotsports.biz/leagues/teamview.asp?teamId=1254&leagueid=144

:)

2 years ago

Ruby 1.8.5, REXML, and You

In my previous post, I mentioned that I was trying to track down why our mongrels (running Rails) at work were spinning out of control for no apparent reason. However, finally being able to generate a backtrace from those mongrels led right to the issue.

The relevant portion of the backtrace looked like this:

from /usr/lib/ruby/1.8/rexml/encoding.rb:59:in `check_encoding’
from /usr/lib/ruby/1.8/rexml/source.rb:40:in `initialize’
…
from /usr/lib/ruby/1.8/rexml/document.rb:45:in `initialize’
…
from ./current/config/../vendor/rails/actionpack/lib/action_controller/cgi_ext/cgi_methods.rb:53:in `parse_formatted_request_parameters’

So apparently REXML was the culprit, and it was happening while trying to parse the parameters that were passed to our Rails app. Before any of our (non-framework) code was even called. From some of my investigation, I knew some of the spinning mongrels corresponded to requests with 200k or more of POST data, so this made some sense. But other requests with that much data went through just fine.

In any case, tracking down that line revealed this code, as part of what REXML uses to detect the encoding of the XML data:

str =~ /^\s*<?xml\s*version=([‘”]).*?\2\s*encoding=([“’])(.*?)\2/um
return $1.upcase if $1

And, well, that’s just wrong. The non-escaped question mark, the two “\2” backreferences, the “return $1” instead of “return $3”, etc. However, our servers are currently running on Ruby 1.8.5. So I took a look at the corresponding code in the version of REXML that comes with Ruby 1.8.6, and found that it’s fixed:

str =~ /^\s*<\?xml\s+version\s*=\s*([‘”]).*?\1\s+encoding\s*=\s*([“’])(.*?)\2/um
return $3.upcase if $3

So it just happens that the sequences of data we were receiving in the rails app were playing unkindly with the broken regular expression, going exponentially out of control with backreferences.

Changing the old REXML code to the new code brought encoding-detection time down to 52 microseconds instead of 2 *hours* on some of the test data. So, we fixed that on our servers, and the problem is gone.

Another solution: Upgrade to Ruby 1.8.6 or better. :)

2 years ago

Backtraces from Live Ruby Processes

We had some out-of-control mongrel processes at work recently… They wouldn’t respond to anything except a “kill -9”, were taking 100% cpu time, and spinning for hours before responding to any new requests.

Unfortunately we had no idea what could be causing it, either. Loading up gdb and printing out a backtrace only gives a bunch of “rb_call” type entries, the C-level code rather than anything at the ruby level. The obvious solution is to go ask Google how to get a backtrace.

Presumed answers:

http://weblog.jamisbuck.org/2006/9/22/inspecting-a-live-ruby-process and http://eigenclass.org/hiki.rb?ruby+live+process+introspection

However, they must have been working with a different version of Ruby, because that doesn’t work on my local Mac or the Linux servers; I just get errors about accessing invalid memory. So after a few frustrating hours I downloaded the source code for Ruby 1.8.6, and found rb_backtrace(). It’s a function that takes no arguments, and prints a backtrace to stdout rather than trying to return an array (which would be semi-difficult to interpret in gdb).

So …

$ gdb -p <process_id>

(gdb) call (int)rb_backtrace()

Forces ruby to print a backtrace out to the process’ stdout. In our case, that’s a daemontools log file, so I could finally track down what was making the mongrel spin forever!

There’s probably still a great way to do this within gdb, so you don’t have to go find the stdout, or possibly get debug data where it shouldn’t be. But it’s a starting point.

2 years ago