Capture network traffic with charles proxy

We want to capture our ios application’s network traffic. This is something we usually do on our physical Mac boxes with a proxy called Charles (www.charlesproxy.com). Charles has a headless mode (non GUI command line mode) that fits perfectly our need.

We do this with the following instructions:

  • Download Charles via brew
  • Install charles root certificate into the simulator’s Keychain
  • Get the VM’s current ip address into $url
  • Start Charles in headless mode on port $url:8888
  • Reconfigure the network via networksetup –setwebproxy $url:8888 and -setsecurewebproxy $url:8888 and export http(s)_proxy environment variables…
  • Run the application in the simulator
  • Dump the traffic through the export feature of Charles rest-api.

This works like a charm on our macs.

The same does not work in the bitrise virtual machines, no errors are raised but there’s no traffic intercepted in the proxy: after a long search, I found out with tcpdump that the simulator does NOT use the system wide network settings, and therefore does not go out through port 8888. The tcpdump shows that the packets go straight out to internet, ignoring the proxy. In other words, the simulator doesn’t honor any proxy settings, neither those or networksetup neither those via the http(s)_proxy env. variables.

Is this something you are aware of?

Do you know of other solutions to capture outgoing traffic from the simulator? I guess using another proxy like burp won’t get us forward, as it looks more like a problem with the simulator not using the network settings…

1 Like

Not really, we don’t change any MacOS settings other than what’s described here: https://github.com/bitrise-io/osx-box-bootstrap , just basic things (like turning off Sleep and Mac automatic updates and similar things).

In short I believe there’s no reason why it would not work on bitrise.io if it would work on a brand new Mac if you do the same steps. Emphasis is on brand new / clean macOS, as these type of issues are usually related to something like you in the past once activated X on your own mac but forgot about it, but X is required to make it work on a clean macOS install.

I’d suggest you to try your setup in a virtual machine after a clean macOS install, if possible, that should give you a similar environment to the one the builds are running on bitrise.io

If there’s no option for that then feel free to contact us via email / onsite chat to request a temporary, user accessible VM (a VM created from the same image as the build VMs but started on a publicly accessible network so that you can access it).

Please let us know if you have any questions and Happy Building! :slight_smile:

Ok we will clean the machine and do it again…

1 Like

Let us know how it goes! :slight_smile:

P.S.: for debugging this might help, this is a quick and dirty guide about how you can connect to the MacOS Build VMs via VNC/Remote Desktop: https://gist.github.com/viktorbenei/c6d4fe1e68de739dbb5f4f15de76b9db

1 Like