Safety First!

Launching weather balloons can be a safe and fun hobby if done correctly. It can also cause serious physical harm or property damage if done recklessly.

Here at the Casa de Balloon Club, we take safety very seriously, and so should you.

Overall Safety Advice

  1. Learn from experienced balloonists before launching your own balloons.
  2. Be paranoid and be conservative.
  3. Stay away from populated areas.
  4. Read and follow FAA regulations.
  5. If you feel unsafe, STOP.

Payload Preparation Tips

  1. Work in a well-ventilated area — both cutting foam and soldering release nasty chemicals into the air, and you should avoid prolonged exposure to them.
  2. Be careful working with Lithium and LiPo batteries — if shorted, they can start fires.
  3. Do not put anything in a payload that you are not willing to part with forever.
  4. Clearly mark your payload as a "Weather Balloon Payload" and include your contact info (phone number and e-mail address) on the outside.
  5. Put a tennis/other ball on the end of any protrusions. You do not want anything pointy on your payload, as they become spinning javelins of death on descent.

FAA Tips


  1. Read the actual regulations — they change constantly, and other resources may be out of date.
  2. Familiarize yourself with SkyVector and make sure your predicted flight path stays well clear of any airports and restricted areas. One good resource on how to read a sectional is this guide, but it's best to ask for help from a licensed pilot.
  3. Call up the Lockheed-Martin NOTAM Filing Service (877-487-6867) the day before your launch and ask to file a HIBAL NOTAM. This warns pilots flying in the area to keep an eye out for your balloons. When you call, have your launch date, launch location coordinates, launch time UTC, and flight forecast prepared and organized based on these directions. The agents are extremely friendly and helpful.

Travel Tips

  1. Travel in groups, never alone.
  2. Pack plenty of water, food, and a first-aid kit — your activities will be in open rural areas where help may be quite far away.
  3. Know where the closest emergency medical facility is and the directions on how to reach it. When you need help, you may not have cell service to look it up or call for help.
  4. Share your travel plans with your friends/family back home so they know when to expect you to check in/be home.
  5. Wear sunscreen.

Launch Planning Tips

  1. Pick a launch site that is well clear (>100 yards in all directions) of telephone poles, power lines, buildings, trees, or any other obstructions. Also stay away from airports (even private ones) and highways. If your balloon is under-inflated it may rise slowly while drifting in the wind, and it can catch on anything sticking out of the ground.
  2. Run launch simulations based on your chosen launch site, and run them again immediately before you launch — make sure that the target landing area is safe and unpopulated.
  3. Vary launch parameters (ascent rate, descent rate, max altitude) by +- 20% and ensure your target landing area is still safe and unpopulated.
  4. Launching in high winds is difficult and dangerous. Launching near sunrise is generally a better time for calmer winds.

Balloon Inflation Tips

  1. Handle the helium tanks with care! This is a good guide.
  2. Helium tanks, nozzles, and inflation tubes will get very cold in the process of being emptied — be careful of frostbite, and avoid touching exposed metal parts.
  3. Keep a hand on the helium valve at all times to regulate the flow of gas, and be able to shut it off quickly if need be. Do not inflate the balloon too quickly, especially at the beginning of the fill.
  4. Make sure the balloon is sufficiently inflated for a speedy ascent (~5 m/s). You want the balloon to ascend quickly to get away from any potential hazards.
  5. Yes, helium is expensive. Yes, hydrogen is cheaper. Remember the Hindenburg? Use helium.

Pavement Ends

Driving Tips

  1. Rural unpaved roads are full of obstacles, potholes, debris, and other challenges — have the driver concentrate 100% exclusively on what's right in front of them and have a second navigator looking at maps/GPS to figure out where to go.
  2. Bring a jumpstarter/inflator just in case. AAA takes a while to make it out to random farm roads.

Recovery Tips

  1. No payload is worth risking your life for. If you are unsure about your safety, STOP. Don't do it.
  2. Be prepared to never see the payload again — do not put something in there so valuable you will take risks to recover it.
  3. Beware of rattlesnakes. Seriously.
  4. Do what the person with the shotgun says. Seriously.
  5. Respect private property owners and ask for permission whenever practical. If asked to leave, do so immediately and do not return.
  6. Do not disturb the area, and be respectful and quick about retrieving the payload.
  7. Bring some bright orange in case your payload lands a recreational hunting area.
  8. If attempting a water recovery, wear a life vest no matter how good a swimmer you think you are.
  9. Be careful around riverbanks and marshy area — mud may be much deeper than you think.
  10. Bring something long to help extend your reach so that you don't have to put yourself in precarious positions to recover a payload.

Helium

You shouldn't inhale helium. It is dangerous — it deprives your body of oxygen and can result in injury or DEATH. Slate has a great Explainer about how helium can affect you.

We know people like to hear funny high pitched voices, so if you are going to do it anyway, be smart:
  1. Never inhale helium from the tank (or any other pressurized source) as it could damage your lungs, killing you instantly. When people from our team choose to inhale helium, they do it by partially filling a practice balloon with excess helium and then inhaling from that balloon.
  2. Do not take more than one breath of helium at a time. After an inhalation, breath air normally for a few minutes.
  3. STOP if you feel light headed!

Team

Vimal Bhalodia
Balloon Father

Vimal not only came up with the idea — he's also the technical mastermind behind the operation.

"I've got this crazy idea..."

Launches 1, 2, 3, 4


Heather Brundage 
Ringmaster

Heather amplifies the awesome and leads the logistics. Whether it's figuring out how to secure the payload, navigate the farm fields, or wrangle the crew — she's got this!

"Let's do it! And...done."

Launches 1, 2, 3, 4



David Blaikie
Stunt Driver

Dave's extra-long arms are deft at guiding the chase car over hill and dale, holding the balloon during filling, and retrieving payloads from rattlesnake-infested swamps.

"Go-Go-Gadget arms!"

Launches 3 & 4



Clare Bayley
Task Master

Clare's authoritative voice guides us through the launch procedure with ease. Also, she brings the big drones.

"Ensure space is clear of anything that could puncture the balloon (including birds....and drones)."

Launches 1 & 4



Peter Alau 
Armored Support Crew 

The Peter action figure comes equipped with motorcycle and full body armor — perfect for braving the hazardous conditions of balloon retrieval. He does all his own stunts. And yours.

"A few more inches and I would have been soaking wet!"

Launch 2




Charlene Justin
Javed Samuel
Balloon Crew
Balloon Crew
Launch 1
Launch 1


Rachel Lancaster
Justin House
Balloon Crew
Balloon Crew
Launch 2
Launch 2


Nancy Yang
Rusty
Photog
Mascot
Launch 2
Launch 4

Tracking

Launching a balloon is only half the challenge. If we want to see the images and video footage the balloon payload captures during its flight, we need to be able to retrieve the payload. And for that we need to know where it lands.

Though the HabHub predictions are impressively accurate, they are not good enough to lead us to the exact landing spot required for retrieval, so we need the payload to be able to tell us where it is.

To Pinpoint the Landing Location


Strobe Light

Depending on your landing area, you could equip your payload with a bright strobe — get close enough to the landing location using estimates, and hone in on the exact location by looking for the strobe.

The challenge with this method is that a strobe light adds weight and requires power. If it is not recovered in a timely manner the power will eventually run out, disabling the strobe. Additionally, if the payload lands with the strobe light buried or blocked, the strobe will not help you locate the payload.

Audible Ping

Another option is for the payload to emit a loud sound so that once you are close by you can hear where the exact location is. Like the strobe solution, this adds weight and consumes power. The noise may also be disruptive or annoying to people near the landing area.

Both of these methods only work for (potentially) finding the location of the balloon when it is near or on the ground — neither allows you to track the balloon in flight.

Cellular Communication

In order to track the balloon in flight and track where it lands, we wrote a tracker app that enables the cell phone in the balloon payload to text us its location and altitude while it has cell reception. Once we receive the text, we map the GPS coordinates using Google Maps. Since we already have a cell phone on the payload, there is minimal addtional power use and no extra weight to have the cell phone text us every minute while it has cell reception. But it means we need to launch and land in areas with good cell phone reception and that we do not know where the balloon is while it is above ~20,000 ft, as it is too high to access the cell network.

Radio Communication

In order to track the balloon when it does not have cell reception, we place a 433MHz radio onboard the payload and use a directional antenna to receive the signal. This method adds some weight and power drain. It also requires that the operators have at least a Technician Class Amateur Radio License, are within range of the payload, and have an unobstructed path between the transmitter and receiver. Once the payload is on the ground, the transmission will most likely be blocked by the ground or other obstructions, so unless you can track the balloon closely during the entire flight, a radio is not ideal for finding the exact landing location.

We did successfully track the balloon using radio during our third launch, and you can read more about that here.

Backup

If all else fails, you may be able to take advantage of the Android Device Manager to locate your balloon. We had do this on our first launch when one of our cell phones never got a valid GPS fix. Luckily, Android Device Manager was able to triangulate and provide a very accurate location of the payload. Though this was a lifesaver, we would not recommend relying on this service as the main tracking method.

Retrieval


Once we get the GPS coordinates of the payload landing site (from radio communication, cellular communication, or Android Device Manager), we have to get to the landing site. This is often one of the most challenging (and fun!) parts of the day.

In order to automate the process of transferring the cell's texts into a Google Map, we created a web app that plots all received location texts from the cell phones. Here are the results from our fourth launch:

Launch 4 balloon tracking.

Though this gives a great overview (and is much more helpful than having to copy and paste GPS coordinates into Google Maps while on the chase), even when zoomed in, Google Maps isn't necessarily helpful in getting you to the exact landing location.

Landing location of Chuva from launch 4.

Since we aim to land in unpopulated areas, we're not landing near marked roads.  Therefore, in order to get close to the landing location, we rely on Google's satellite imagery and real-time problem solving to get as close as possible to the payload.


Luckily, Chuva landed right alongside a dirt road, and we could drive up pretty close to retrieve.

Other times we have had to walk long distances or find ways around unanticipated obstacles.  Always be careful and respectful when recovering your payload!

Building the Casa de Balloon Tracker App

In order to track our weather balloon payloads and capture near-space photos, the Casa de Balloon Club uses Android phones running our own tracker app.


For those of you interested in making simple Android apps such as this one, we made our app source code available — feel free to clone, fork, and tweak!

Background


When we started on our ballooning adventures, we decided our very first payload would be a simple Android phone — after all, it has GPS, a camera, and a radio, and we would not be too upset over losing $50 worth of electronics.

We were hoping to use one of the many GPS tracking and camera time lapse apps already available in the Play Store, but unfortunately none of them worked precisely the way we wanted, so we decided to make our own.

Hello World!


None of us had ever developed an Android app before, so the obvious first place to start was with the Getting Started tutorials.

This was surprisingly easy — we had a skeleton app up and running on a cheap Android phone with just an afternoon of hacking.

We deliberately set our application to target API Level 15 (Android 4.0.4) because that API level had all the features we wanted and allowed us to run the same app on a variety of old / disposable hardware.

MainActivity


We tweaked the screen from the first tutorial tutorial into the main control screen for our app:


This includes some basic config display/settings capability as well as buttons to test specific functions of the app such as sending a location SMS or taking a photo.

MainService


Right from the start, we knew our app would need a background service which would periodically text GPS coordinates to a specific number.

After getting a simple app running, we skipped the rest of the Android developer tutorials and went straight to the section on creating background jobs.

One of the unique and extraordinary things about Android is the power of background services — they literally can do anything that a normal app can do with no random constraints. GPS, HTTP, SMS, photos — our background service could do whatever it wanted!

Location Listeners


The next thing we decided to work on was location services, which was surprisingly complicated but extensively documented.

Most of the location-related documentation talks about different strategies for finding location and tradeoffs in terms of accuracy and power consumption. For our application, the goal was to have as accurate and up-to-date a location as possible, and we didn't care about burning power in the process because we had plenty of battery life.

Even though we only needed location updates every ~30 seconds to geotag our photos, we learned that if there was too great an interval between requested updates, the GPS location provider would lose the fix and not always reacquire it fast enough. Thus we added a noop location listener which subscribed to location updates every 150ms, forcing the GPS chip to maintain a continuous fix.

We also added an NmeaListener which logged the GPS coordinates to a file in a standard format parseable by lots of mapping tools.

Status SMS


Now that we were getting GPS coordinates, it was time to send them back to us via SMS. A simple Timer kicked off in MainService triggered a static sendLocationSMS utility function which fetched the latest GPS coordinates, converted them to a clickable Google Maps URL, and sent them to us.

The SmsManager docs were a bit beginner-unfriendly, but the Android developer community came to the rescue with a simple example.

Since we had a couple extra characters left in our SMS payload, we decided to add battery voltage courtesy of the BatteryManager.

Photo Activity


With the tracking function up and running, the next question was how to take some photos. We experimented with many different code snippets to take a photo, but learned that some Android devices (like the Nexus 5) refused to take a photo unless a live preview to an active canvas was enabled.

Thus we decided to create a single-purpose photo activity which would initiate a photo capture on startup and then quit after capture success.

The Android Camera API is quite sophisticated, and we really dug into and tweaked most of the available settings in order to make sure we were taking the best pictures we could.

A couple of the more important parameters we set include:

  • Flash off
  • Focus at infinity (for high-altitude landscape photos there is no point trying to autofocus)
  • "Action" or "Sports" scene modes to favor higher shutter speeds over lower ISOs and prevent our pics from being blurry due to balloon movement
  • Highest JPEG quality — we were using external SD cards with plenty of storage space
  • EXIF GPS and Timestamp info if it was available from location services

Under normal use, we had this activity triggered by an AlarmReceiver which itself was triggered at regular intervals by an AlarmManager. Android Alarms are another fascinating topic with lots of relevant documentation.

Other Fun Notes


In later versions of the app, we added support for periodically uploading the current location to tracker.casadeballoon.club, giving us live real-time tracking of portions of the balloon ascent and descent where there was cell coverage.

We also spent some time reading about different Android power management options to ensure that our phones never went to sleep while they were engaged in very important background tasks.

In the next versions of the app, we hope to play with the motion sensors APIs and find a way to time the snapping of our photos to when the payload is aligned and relatively still.


Android at 100,000 ft

The GPS on pretty much every Android phone deliberately stops working above 60,000 feet.

This is the story of why and how we hacked around that limit and got our Android powered Nexus S to maintain a GPS lock up to 100,000 feet above the ground.




Note that we're not going to distribute any modified binaries, since we are not licensed to do so. But we will explain exactly what we did in sufficient detail that you should be able to replicate our results.

Table of Contents:



Motivation:


For amateur weather balloon enthusiasts, a used Android phone makes the perfect tracking device — it has a radio, a camera, sensors, GPS, storage, and a pretty powerful CPU all wrapped up in a nice package you can buy for <$50 on eBay.

The only downside is that the GPS stops working above 60,000 feet, while a weather balloon launch typically goes well above 80,000 feet and sometimes above 100,000 feet. This means the GPS logs from our phones end up looking like this:


The failure of GPS is not a technical limitation — it’s a legal one. Specifically, a 1960’s era piece of regulation introduced the “CoCom Limits” which require that all commercial GPS chipsets disable tracking if they determine that they’re traveling above 60,000 feet and above 1200 mph. The original reason was to prevent no-goodniks from using commercial GPS units to make ICBM guidance systems.

As written, the CoCom limits are perfectly reasonable and do NOT interfere with our weather balloon adventures, which meander along at a leisurely 75 mph. Unfortunately, the limits as written are not the limits as implemented — specifically most commercial GPS implementations inside phones will be disabled if they go above 60,000 ft OR above 1200 mph.

If only there was a way to poke inside the GPS logic and turn that “OR” into an “AND”...



Exploration:


When we first ran into the 60,000-ft limit on our Android phones, we downloaded the Android source code and poked around a bit in the location services and GPS system to see if we could find any code that enforced these limits. Unfortunately, no luck — all we learned is that the the GPS service interfaces with the underlying hardware via an NMEA-like interface, which pre-filters location data.

But digging into the wonderful Android developer community at xda-developers, we stumbled across two amazing posts:
After reading through both threads, we learned two critical facts:
  • Broadcom GPS chips offloaded the actual position calculation to a regular arm-linux binary running on the host Android system (glgps / gpsd)
  • That binary has extensive debug-level logging statements built in
Since we were using a Nexus S phone which has one of these Broadcom GPS chips in it, we decided it was time to nerd up and learn how to reverse engineer and make a binary modification to a closed-source executable. Wheeeee!

Rather than writing a book about all the different things we tried which ended up going nowhere, we’ll instead focus on the much shorter set of steps we took which ultimately ended up working and why they were necessary.



Setting up Android:


We started with installing CyanogenMod on our Android phone.

Like us, you may have thought Android is just Linux, which would mean once you got a shell, you’d be ready to ls cp mv cat | grep. That’s only half-true — stock Android runs the Linux kernel and has a shell, but that shell doesn't quite work like the standard bash shell we're used to.

To get all the Linux shell goodies we actually wanted, we would have had to go through some sort of ugly process of rooting our phone, swapping out the default Android shell for busybox, and installing a bunch of other utilities which we would expect to be present on any Linux system we’re working on.

Fortunately, because of Android’s open-sourceness and the awesomeness of the CyanogenMod team, we were able to just download and install a stable, high-quality ROM that had all of the developer features and utilities we needed installed and configured correctly.



Extracting the binary gpsd:


Per the xda-developers forum threads, the binary of interest on our Nexus S phone was located at:
/system/vendor/bin/gpsd

In order to get to it, we first restarted ADB as root and then pulled a copy of gpsd and its config file to our local ubuntu desktop:

-- Restart ADB as root on the target device --
ubuntu:~/$ adb root

-- Remount /system to be read-write to change system files --
ubuntu:~/$ adb shell "mount -o rw,remount /system"

-- Pull the gpsd binary and its corresponding config file --
ubuntu:~/$ adb pull /system/vendor/bin/gpsd
ubuntu:~/$ adb pull /system/vendor/etc/gps.xml


As a first exercise, we decided to change the logging settings in gps.xml and see if we could get some verbose debugging output. Here is our modified gps.xml.

After changing gps.xml locally, we had to push it to see if it worked:

-- Push the edited gps.xml --
ubuntu:~/$ adb push gps.xml /system/vendor/etc/gps.xml

-- Kill the currently running gpsd process, forcing it to restart and read the new config file --
ubuntu:~/$ adb shell "killall gpsd"


Then we started up the GPS Test app, waited for a bit, quit out, and downloaded the log file:

-- Pull the gps log file --
ubuntu:~/$ adb pull /data/gps/log

-- Peek at the gps log file --
ubuntu:~/$ less gl-*.txt


The giant dump of log messages meant we successfully managed to replicate the work of everyone else who has been hacking on Android and Android GPS to date, which serves as an excellent starting point.



Reverse-engineering gpsd part I: Disassembly


The first question we had was: "Are there any log messages in gpsd related to CoCom altitude limits?”

ubuntu:~/$ strings gpsd | grep -i limit

CP_W: %s: SatID:%d out of the limits
<SKFNAV> COCOM limit exceeded. Alt=%em No output & KF reset!
<SKFNAV> <SKFNAV> Speed limit exceeded. Speed=%1.0f m/s No output
%sSetUserDynamicLimits(%u m/s, %f m/s^2)
UpdateTimeStamps: state %s curr %lu Stat %ul Driv %ul Train %ul ...
Binwmm::mode3 (ch %d sv %d) bin limit %d reached - reporting no energy
Binwmm::mode3 (ch %d sv %d) retry limit reached - reporting no energy
Binwmm::mode3 (ch %d sv %d) retry limit not reached (cnt = %hu) - returning to coarse stage
toeLimit
'pvalue->toeLimit'


Jackpot! There was a nice, simple, straightforward log statement pointing to exactly what we wanted. Thank you, Broadcom!

Had we not found such a statement, you probably would not be reading this article because we would have given up and spent the rest of the evening watching cat videos on YouTube.

With log statement in hand, all we had to do is find the code path which referenced it and insert a couple strategic NOOPs. Easy peasy lemon-squeezy, right?

Having no desire to trace through lines of raw ARM assembly, we looked online for good ARM disassemblers and decompilers which could produce a C-like output. Some of the options we encountered included:
  • IDA Pro - Looked awesome, but the trial version was sufficiently crippled to not be usable for this project, and the license was outside our budget for something we were only going to use once.
  • Retargetable Decompiler - Looked like it could have been perfect, but their online demo barfed on our gpsd upload.
  • Hopper - Claimed to have all the bells and whistles that we wanted, offered a very usable trial version, and had a perfectly affordable license cost of $89.
We downloaded the Hopper trial, and after poking around with it for a couple hours, we were sufficiently pleased with it to justify paying the $89 and purchasing a license.

We fed gpsd into Hopper, and after a few minutes of chugging, it gave us a nice searchable view of the assembly code:



The first thing we did was search for the debug statement, and we found the address for the start of that string in the .data section at address 0x00156b52:



The next thing we did was search for any references to 0x00156b52 anywhere else in the assembly. Not found — uh oh! This meant one of two things: either the debug statement was dead code not used anywhere, or there was some other mechanism for referencing string constants in print statements.

To dig into this a bit further, we decided to switch to one of the debug statements from our log file:
14:37:37.940 #372590D <SKFNAV> SKF_AUTO!!!

We found the corresponding string constant at 0x00157851. That address was also not referenced directly anywhere in the code, which was good news and bad news.

The good news was that clearly there was some other method of referencing string constants.

The bad news was that now we either had to trace through a truckload of assembly to figure out what that method was, or we’d have to learn how to use a binary debugger to narrow down the amount of assembly we had to paw through. Both of those options sounded strictly less fun than watching cat videos.



Reverse-engineering gpsd part II: Debugging Setup


Having just enjoyed a refreshing evening of cat videos, we once again decided to nerd up and learn us some gdb.

We were not quite sure where to begin, but RMS came to the rescue with this amazingly well written and accessible GDB tutorial.

With some basic knowledge about how to use GDB under our belt, it was time to learn how to run gbdserver / gdb on Android.

There were actually many many more tutorials on how to use gdb both in general and on Android, but most of the "simple" ones were useless because they all were based on the assumption that you were trying to debug a binary which you wrote and compiled, with debugging symbols which mapped back to your source code. Whereas we were working with a stripped binary, no source code, and a vague decade-old understanding of assembly language, stack traces, and core dumps.

We got gpsd running under gdbserver by doing the following:

-- Move gpsd to gpsd.original to prevent it from auto-starting --
ubuntu:~/$ adb shell
root@android:/ # cd /system/vendor/bin
root@android:/ # mv gpsd gpsd.original

-- Kill existing gpsd process and restart under gdbserver --
root@android:/ # killall gpsd
root@android:/ # gdbserver :5039 gpsd.original -c /vendor/etc/gps.xml

-- Enable gdbserver port forwarding --
ubuntu:~/$ adb forward tcp:5039 tcp:5039

-- Start gdb --
ubuntu:~/$ $ANDROID_HOME/android-ndk-r10d/toolchains/arm-linux-androideabi-4.9/prebuilt/linux-x86_64/bin/arm-linux-androideabi-gdb gpsd

-- Connect to the gdbserver instance --
(gdb) target remote :5039

-- Run gpsd and make sure it still works --
(gdb) continue

Then we fired up the old faithful GPS Test App and made sure everything was still working.



Reverse-engineering gpsd part III: Debugging Debug Messages


Now that we had a functioning binary debugging setup, it was time to figure out just how in the heck those debug print statements worked. After reading through the gdb tutorial, we decided the easiest solution was to set a read watchpoint on the memory address of a debug statement that appeared in our logfile:

(gdb) rwatch *0x00157851
Hardware read watchpoint 1: *0x00157851
(gdb) continue
Continuing.
Cannot access memory at address 0x0
Warning:
Could not insert hardware watchpoint 1.
Could not insert hardware breakpoints:
You may have requested too many hardware breakpoints/watchpoints.


Unfortunately, that seemed to do a whole lot of nothing. After a bit more digging, we learned that read watchpoints are an extremely CPU/architecture-specific feature, and in the case of ARM you only got them if you were using a hardware JTAG debugger. We briefly toyed with the idea of busting out the soldering iron and logic analyzer before coming to our senses and deciding not to shave that particular yak.

Instead, we resigned ourselves to working with function pointers instead. Switching back to Hopper, we searched for a printf-like function and found sprintf at 0x00014bc8.


We decided to stick a breakpoint on it and see what happened:

(gdb) b *0x00014bc8
Breakpoint 1 at 0x14bc8
(gdb) continue
Continuing.
Breakpoint 1, 0x00014bc8 in sprintf ()

Success! We hit the breakpoint. Without going into too much detail, this particular sprintf turned out to be a dead end. So we went ahead and ran to the next sprintf breakpoint:

(gdb) continue
Continuing.
Breakpoint 1, 0x00014bc8 in sprintf ()

Now it was time to look at the registers and see if any of them held pointers to anything that looked like a valid debug string address:

(gdb) info registers
r0             0xbefb95dc       -1090808356
r1             0x150123 1376547
r2             0x197cad 1670317
r3             0xbefb96dc       -1090808100
r4             0x196b48 1665864
r5             0xbefb95dc       -1090808356
r6             0x197cad 1670317
r7             0xbefb96dc       -1090808100
r8             0x0      0
r9             0x0      0
r10            0x0      0
r11            0x0      0
r12            0x0      0
sp             0xbefb95d0       0xbefb95d0
lr             0x232f3  144115
pc             0x14bc8  0x14bc8 <sprintf>
cpsr           0x80070010       -2147024880


According to the decompiler, the string being printed should be in either r0 or r1. Sure enough, the value 0x150123 in r1 pointed to the string constant "%s/%s" in the data section.

Furthermore, to confirm earlier findings, a search for that string constant address in the assembly yielded nothing, meaning it was computed on the fly in some previous function.

So next step was to walk up the stack trace, taking a look at each function to determine where the string address calculation happened:

(gdb) backtrace
#0  0x00014bc8 in sprintf ()
#1  0x000232f2 in ?? ()
#2  0x000233b8 in ?? ()
#3  0x0002348a in ?? ()
#4  0x0001e826 in ?? ()
#5  0x0001e8d6 in ?? ()
#6  0x0001deaa in ?? ()
#7  0x00015a3c in ?? ()
#8  0x00015180 in ?? ()
#9  0x00014f34 in ?? ()
#10 0x000150c0 in main ()


Looks like this sprintf returns to address 0x000232f2 which means it was invoked at 0x000232ee.

Let's decompile this function and see what it does:



Oh, wow, isn't this convenient. The r1 argument to sprintf is calculated by adding whatever value is in r3 to a constant 0x0019bb48. Perhaps this means string references are stored as offsets?

Armed with this assumption, the next question was: could we find which codepath triggered our "altitude limit exceeded" debug message? Its address of 0x00156b52 corresponded to an offset of 0xFFFC000A from 0x00196b48, so we searched for “0xFFFC000A” in the assembly code...and bam!


This reference to our string constant was contained within a function at address 0x00048078 (sub_48078). When we decompiled that function, we saw a nice “if(something) { barf }” structure:


Just to confirm, we took a look at the second structure and saw it reference an offset of 0xfffc0048, which corresponded to a string address of 0x00156b90, which held the debug message "<SKFNAV> <SKFNAV> Speed limit exceeded. Speed=%1.0f m/s No output "



Reverse-engineering gpsd part IV: Binary Patching


So it looks like the function sub_48078 was some sort of CoCom limit check. Furthermore, it looks like function sub_48078 is only invoked one time at address 0x0004e6a6:


We decompiled the function which called sub_48078, and saw a bunch of other debug messages related to various error checks:


So it seemed to be a pretty safe bet that if we NOOP the invocation to the CoCom check at 0x0004e6a6, then our GPS should happily function above 60,000 feet!

Hopper made this surprisingly easy — with just a couple clicks, we disabled the function invocation at 0x0004e6a6 and exported a new binary.



For those of you following along at home with your own disassembler and debugger, here’s some helpful info:

ubuntu:~/$ md5sum gpsd.original
4a6c0027e530b5b8a346153a355ef8e3  gpsd.original

ubuntu:~$ md5sum gpsd.modified
77808f7c84f60da2ca494c81b369d77b  gpsd.modified

ubuntu:~$ cmp -l gpsd.original gpsd.modified | gawk '{printf "%08X %02X %02X\n", $1, strtonum(0$2), strtonum(0$3)}'

000466A7 F9 00
000466A8 F7 BF
000466A9 E7 00
000466AA FC BF

Time to push this patched gpsd to the phone, restart gpsd, and make sure it all still works:

ubuntu:~/$ adb push gpsd.modified /system/vendor/bin/gpsd.modified
ubuntu:~/$ adb shell
root@android:/ # cd /system/vendor/bin

root@android:/ # chmod 755 gpsd.modified
root@android:/ # gpsd.modified -c /vendor/etc/gps.xml


When we fire up our favorite GPS Test App, we continue to get a GPS fix — so it seems like at the very least our modified binary didn’t completely break everything!

May as well make our patched version the default gpsd:

root@android:/ # cd /system/vendor/bin
root@android:/ # cp gpsd.modified gpsd




Testing part I: GPS Simulators


A couple of our awesome friends — specifically Eric Jonas at Berkeley and Jeremy Conrad at Lemnos Labs — put us in touch with Fergus Noble, the founder and CTO of Swift Navigation (a company which makes high-precision GPS equipment, which I hope one day finds its way into all of our phones).

Fergus was gracious enough to spend an afternoon giving me a tour of Swift Navigation and talking shop about what they were working on. It’s pretty awesome, and I highly recommend you take some time to check them out — http://www.swiftnav.com/. He also hooked me up with their Spirent STR4500 GPS simulator so that I could sit comfortably at a lab bench with my android dev setup and trick my phone into thinking it was on a cross-country airplane flight,  just to see if my modified gpsd was still working. Fortunately, everything looked like it was working perfectly — woohoo!

Fun Fergus Fact: He was president of Cambridge University Spaceflight. If that sounds familiar, it's because it's the group which developed some of the best resources available to amateur weather balloon enthusiasts.



Testing part II: Real Life


Unfortunately, the GPS simulator was unable to replicate scenarios above the CoCom limits. We figured the next best option was to spin up some sort of Felix Baumgartner / Alan Eustace-style operation so we could title this article “Debugging Android at 100,000 ft”.

Sadly, the time and cost of such testing exceeded our project budget by two orders of magnitude, so we followed in the footsteps of thousands of product managers in similar positions by declaring, “Screw testing. Let’s ship!”


With the help of the rest of the Casa de Balloon Club and some extra special love from the "Made With Android" team at Google, we shoved our modified Nexus S into a foam box, tied it to a helium-filled weather balloon, and let it loose over the Central Valley.

And to everyone’s surprise, we not only recovered the payloads and got some awesome pictures, but we also discovered that our modification did in fact have the desired effect and our Android phones successfully tracked our entire flights!



Epilogue


In that launch, we actually sent up both a modified Nexus S as well as a modified Samsung Galaxy Camera GC100, both of which used Broadcom GPS chipsets and both of which worked swimmingly.

In future launches, we plan on sending up a modified Samsung Galaxy SIII i9300, which has much newer hardware but continues to use the same Broadcom GPS chipset.

If you happen to be interested in replicating this project I highly recommend purchasing a used Nexus S. But if you’re up for doing some reverse engineering of your own, a quick crawl through all of the CyanogenMod device repos shows that the following supported devices all have Broadcom GPSs and should be hackable following the same process:

  • captivatemtd - Samsung Captivate
  • crespo - Nexus S
  • galaxysmtd - Samsung Galaxy S i9000
  • i9300 - Samsung Galaxy S3 i9300
  • i9500 - Samsung Galaxy S4 i9500
  • n7000 - Samsung Galaxy Note n7000
  • n7100 - Samsung Galaxy Note 2 n7100
  • superior - Samsung Galaxy Premier i9260
  • vibrantmtd - Samsung Vibrant
  • enrc2b - HTC One X+
  • p720 - LG Optimus 3D Max P720
  • p760 - LG Optimus L9 P760
  • p970 - LG Optimus Black P970
  • p990 - LG Optimus 2X P990

Also, many older devices using Exynos or Tegra chipsets (such as the Galaxy Camera GC100) are paired with Broadcom GPSs . As long as you can root your ROM and you can find the gpsd or glgps binary, there’s a good chance you can modify it.

Good luck and happy hacking!



Location

After deciding what we were doing to launch weather balloons, our next question was: Where?

Since we are based in San Francisco, we knew we would need to travel outside the city for safe air space.

An ideal space is:
  • Safe: The launch and recovery sites need to be clear, open, sparsely populated, and away from restricted airspace. Though our balloons are exempt from the FAA regulations due to their low weight, we still aim to meet the requirements when looking for a good launch spot.
  • Accessible: The launch and recovery sites need to have access roads. So preferably not in mountains, bodies of waters, or wilderness areas.
  • Connected: Without cellular reception, the payload will not be able to phone home and tell us where it is. We aim to recover every payload we send out, so they need to have solid communication channels.
  • Close: We need to be able to drive to the location and back to San Francisco in a day.
  • Predictable: Clear skies, no rain, regular wind patterns.

In order to find such a spot, we used a variety of resources:
  • Friends who had done this before, who suggested suitable areas.
  • Google Maps to check the locations of roads, cities, mountains, and bodies of water.
  • SkyVector to look at restricted airspace.
  • HabHub to get wind predictions for various launch spots.
  • AT&T coverage map to determine cellular reception.

We discovered that the Central Valley was a perfect launch location! Not only is it ~2-3 hour drive from San Francisco, but it also has large areas that are sparsely populated yet crisscrossed with access roads, mild weather with predictable NW->SE winds, and solid cell coverage.

Central Valley.png
The Central Valley of California
Sky Vector.png
SkyVector Map of the Central Valley

Prediction.png
Typical Central Valley Flight Path

AT&T Wireless Coverage in the Central Valley

A major component of selecting a launch site is determining the landing site. The habhub.org predictions aren’t perfect, but they are impressively close. We want to avoid landing on highways, in cities, in bodies of water, or in the mountains.

So we look for launch sites that result in landing areas like this:

GoodLandingArea.png

When the wind predictions show undesirable flight paths (like the ones below), we delay the launch or pick a different launch site.

BadPrediction.png
Avoid landing in mountains, where it is hard to access the landing site.

BedPrediction2.png
Avoid flying over metro areas and airports.

Here are flight predictions from two of our launches to show good launch and landing areas:

PredictionD.png
Launch 2, abnormal winds for the Central Valley
Chuva.png
Launch 4, more typical winds


The Central Valley is mostly devoted to farming, so we end up landing in various fields.






We also got to experience the California drought up close and personal. You can read more about that in the Where's the Water? blogpost.



To see more pictures of our launch and landing locations, check out our Picasa albums.

Where's the Water?

California is on its fourth year of severe drought conditions, and water reserves are at all-time lows.

The Central Valley is the heart of California's agriculture and has been particularly hard-hit. Our balloon launch excursions gave us a unique, first-hand look at the impact of the drought.

One striking indicator was the difference between the Google satellite images of the landing site and what we found when we got there:

Google satellite image of landing area — a large marsh.

Traversing the former marsh easily, now that it is completely dry. #NoFilter

Though the drought conditions made it much easier to recover our payload (no one wants to tromp through a swamp), the drastic difference made a huge impression on us.

We encountered similar discrepancies again and again, with many of our balloon launch and recovery areas expected to be active farmland or swampy marshland, but instead ending up being fallow fields or dry dirt patches.

Many of our payloads land in fallow fields.

Sneak peek at the Mad Max future.

On our way to Los Banos, we often drive by the San Luis Reservoir. We could clearly see just how low the water was relative to historic levels.

Where did the water go?

Living in the city makes it easy to ignore the drought warnings, but seeing it up close and personal really hit home. It was also eye opening to read the political signs posted around the Central Valley, a very different political climate from the Bay Area.

Political signs around the Central Valley, via @raulatilio