Friday, December 30th, 2016 at 9:44 pm - - Flightlogger

Not been taking time to blog much. I have been lost in a vast pool of non-working projects, and trying to learn scipy and pandas and jupyter, which is an effective replacement for twistcodewiki, which itself was derived from Scraperwiki. The in-browser interactive programming environment Jupyter is everything that Scraperwiki was supposed to have become (eg see data journalism example). But that didn’t happen because every other software engineer I knew assured me in No Uncertain Terms that nobody was ever going suffer the sheer excruciating pain of coding in the browser — when real programmers all want to use vi and git.

But I digress to a world where practically all of my coding is in the browser and I’m using tools like matplotlib at a cycle time of around 10 seconds, which is about a quarter of the time it takes to lose any image file you have generated somewhere on your goddamn disk!

Today I put my new hang-glider data logger (which is held together with lots of glue) into the floor of the elevator and recorded the accelerometer in Z sampling at 100Hz.


With my new powers of filtering

b, a = scipy.signal.butter(3, 0.005, 'low')
pZ["fz"] = scipy.signal.lfilter(b, a,

we can get this:


The art of using sensors is to use a set of them in order to access the underlying reality.

Here is the raw barometer reading as the elevator goes up and down several floors.

Once again it’s a good idea to filter it, only this time we set the frequency of that butterworth filter to 0.01 instead of 0.005 because the barometer readings are at 50Hz, and we need the time delay from the filters to be comparable.


Because it’s nice and smooth after filtering, we can differentiate it by differencing it, using the pF[“df”] = pF.f.diff()*50/10. The times 50 is because it’s at 50Hz, and the divide by 10 is to convert from barometric readings to metres (approximately). The scale is metres per second, and metres per second squared. The elevator travels at about 1.5m/s. The section on the right with the green line lifted for 12 seconds was traveling down about 3 floors at once, so the air pressure was constantly rising.


Acceleration is the differential of the velocity, so setting ddf = df.diff()*50 gets us the following plot:

What we now see are just horrendous oscillations in the barometer. Every time you differentiate you magnify the noise.

But we can reduce the lowpass frequency of this filter by a factor of 5, so it’s 0.001 for the accelerometer and 0.002 for the barometer, and get a curve that agrees (where all the scaling seems to work out without any fudging).


The problem now is you can’t see the elevator moving between the different floors at all. What are those barometric oscillations from? Was I breathing too hard?

Let’s plot the barometer differences against the filtered barometer differences.

There’s a little too much noise, so filter the barometer difference by 0.1 instead and plot it against the 0.01 filter.


No, I can’t see those oscillations in the underlying data either.

To my eyes they only start to spike up with a filter low pass frequency of 0.03.


So I’m pretty stumped. But now I’ve got my eye on it, I can see a slight waviness at about this frequency in the original barometric reading in the third picture from the top, so it’s not an illusion or a creation of this filter.

If I hadn’t just glued my entire unit together I might have separated just this device out to see if it’s caused by yet another oscillation in the circuitry, like the kind I’d seen on a temperature sensor.

Maybe I’ll have to rerun this elevator experiment with all the other sensors turned off as much as possible to see if I can get the the bottom of this mysterious 4 second oscillation in the barometer.

Then there’s the temperature decay curves to sort out (from the use of a hairdryer this morning), which are not as good as I had hoped.

I am pleased to have finally implemented an AB Butterworth filter on my arduino.

It’s not a big deal. The code is roughly as follows, and you can get the a[] and b[] arrays from SciPy by executing the function scipy.signal.butter(3, freq, ‘low’):

class ABFilter
    float* a, *b, *xybuff; 
    int n, xybuffpos; 

ABFilter::ABFilter(const float* la, const float* lb) :
    a(la), b(lb), n(len(la)), xybuffpos(0) // len(a) == len(b)
{  xybuff = (float*)malloc(2*n*sizeof(float));  }

float ABFilter::filt(float x)
    xybuff[xybuffpos] = x; 
    int j = xybuffpos; 
    float y = 0; 
    for (int i = 0; i < n; i++) {
        y += xybuff[j]*b[i]; 
        if (i != 0)
            y -= xybuff[j+n]*a[i]; 
        if (j == 0)
            j = n; 
        j -= 1; 
    if (a[0] != 1)
        y /= a[0]; 
    xybuff[xybuffpos+n] = y; 
    xybuffpos += 1;
    if (xybuffpos == n)
        xybuffpos = 0; 
    return y; 

Friday, November 18th, 2016 at 11:23 pm - - Science Fiction, Whipping

Well, it happened. Donald Trump got elected. And into the hard vacuum of his political philosophy, he’s sucked in all kinds of plague carrying rats.

Rats like Myron Ebell.

People ask me if I am going to restart my proxy blog for Ebell, which I wrote from 2004 to 2011, called The Myron Ebell Climate chronicling his part in the suicide of the human species.

But there’s no point.

Not only is it too late, but everyone else is covering this walking cancer now.

Where were you folks when it mattered? Back in 2004 I said we should run a proxy blog for each one of these think-tank bastards and form a shadow network for this corporate funded disinformation infrastructure. It was required to correct the reputations of these monsters who were constantly popping up on the TV and in the newspapers carrying out lies and damaging all of society. It would have been easy. They’re slippery pricks, but one could systematically keep their record alive to make sure some of it sticks.

Saturday, October 15th, 2016 at 2:20 pm - - Cave, Hang-glide, Kayak Dive

Maybe I’ve got writer’s block. I’ve not even filled these into my logbook. I call it a hat-trick if I do a cave trip, a hang-glider flight and a dive in the same week. This is the fourth time I’ve done it. Generally speaking, the individual events are not all the greatest: the dive was pretty murky, the cave was gritty, and the flight was ridgy. Can’t complain.

The wreck of the Azmund is in Holyhead harbour about a mile of paddling out from the beach.

It was dark and murky and we didn’t find the way back to the boilers after starting on it. The wreck is huge though. Part of the metal juts out of water at low tide.

On the way back we discovered why the beach we launched from is not popular with kayakers — it dries out to about 500m. We couldn’t see our boats after the first time we walked back with a load to the car.

That was Saturday. There was a pleasant day out at Moelfre, with some people being terrified of the currents, but it was the wash from the joy-riding lifeboats that nearly sunk us. The image of the almost breaking 4 metre high wall of water that came upon us while we were anchored in the shallows of Rat Island a few minutes after they zoomed through the channel is going to live long in my memory. The second dive worked out well when we found the remains of the Royal Charter in the sand after groveling in the shallows among the kelp where it was supposed to be until giving up.

Then there was a cave trip to the far end of Ingleborough Show Cave (the only photo of which I have is a line of cavers getting changed on the footpath), followed by a quick escape home ostensibly to start clearing out the house, but which was in fact an excuse to be in North Wales for a flight off Penmaenbach.

I landed on the dwindling beach at high tide after an hour of very smooth sailing in the sea air.


A concrete breaker was hired to smash up and take down the floor. We filled a skip with the crap a day later with some help from friends.

Now we live in a building site. Again. And it’s mid-way through October.

Saturday, September 24th, 2016 at 1:51 pm - - Flightlogger

I’ve been flying around with my data logger for two years now and I have only this week developed a theory of how to process the temperature data. I might have got to it sooner had I known enough physics or bothered to test the actual response time of my Dallas temperature sensor, which was so abysmal I had to start doing something about it. To be fair, it did take me over a year to even shelter it from the sunlight.

Part 1: The responsiveness of the temperature sensor is 0.09

The apparatus involved running a vacuum cleaner against the sensor housing to create a fast airflow while drawing the air either from the room or down a tube containing a bag of ice.


This was the graph it made, holding the intake near the ice and away from the ice for about 40seconds at a time

The curves you see are exponential decay curves, as the temperature sensor cools to the air cold temperature, then warms to the room air temperature, quickly at first, and then slowly as it converges.


Tuesday, September 13th, 2016 at 8:25 am - - Uncategorized

Sensible Code

It’s at the Sensible Code Company whose webpage URL is actually, even though it’s not in the Indian Ocean.

Company was formerly called Scraperwiki of

Nobody ever called my coding Sensible. Things have obviously got to change.

Sunday, September 4th, 2016 at 8:52 pm - - Canyon, Hang-glide

A bit of a bonus holiday slotted in here, with not much time or care to waste on the internet.

With three of us in the car, first we went down to a campsite near Perpignan and picked a 3.6/4 grade canyon of Llech, which was an absolute hit with all of us. There were jumps and toboggans galore, including one called the pistol that shot you up in the air 5m over a deep pool. We need to go back to that area and do the others.


Then I snuck in three days of hang-gliding off Ager with the bonus help of a retrieve driver. On day2 I tried to go behind the ridge, and it didn’t go well. I don’t know how I pulled it down into the field not of my choice without busting something more than the skin of my knee, but I got away with it.

It should have scared me more than it did. Instead my worst moments of terror were on the K5 rated via ferrata Cast Urquiza Olmo to the west of Ager by the lake.

On the third day up the hill I got an amazingly detailed briefing from some visiting British hang-glider named Al about where to go and what to expect (a huge zone of downwash behind the ridge which you can confidently get through if you start with 7000feet of altitude). It worked wonderfully on my third attempt.


Then I let myself down by nearly parking my glider on a rebar fence by curving round too wide and falling short of the field I was aiming for.
But before that happened, the feeling came over me again where my wings became part of me for those few hours up in the sky. The joy was existential. A lot of the time I was circling with vultures. There are a lot of them about.

In the evening we picked up Becka from Lleida and then went to Torla-Ordesa to see the scenery. Unfortunately the morning buses were all full, and you’re not allowed to cycle up the tarmac road which carries the diesel buses because it’s a Spanish national park, so everything is banned even if it doesn’t make any sense. So we walked in until my foot-soles wore out on the way back and I had to catch the bus.


Then, on the way back to France, we dropped in on the 3.2/4 rated Neste de Saux canyon, which wasn’t that great, but we were pleased because we proved we were still up to it.

Now we’re in a gite in Aspet with a bunch of cavers. Everyone is going out to run around the maze known as Felix Trombe, while I have carried on working on the ground window software, where I have implemented a real hack to create flat shading within a the GLSL GPU shading framework:

The trick is to note that the passage tube structures are made from quads which you can colour with 8 colours {-1,1,-2,2,-3,3,-4,4} such that no quad shares a colour with any of its neighbours (edge or corner otherwise). Choose the brightness factor on each quad (say it’s 0which goes:

float flatfac = max(0.0, max(max(abs(vflat4.x), abs(vflat4.y)), 
                         max(abs(vflat4.z), abs(vflat4.w))) - 510.0); 
gl_FragColor = mix(gl_FragColor, vec4(0.0, 0.0, 0.0, 1.0), flatfac); 


(I’m now fed up with fighting with this crappy internet connection. It’s probably why I’m getting so much work done.)

Wednesday, August 24th, 2016 at 9:16 am - - Hang-glide

I spent the last three days at the washout known as the British Open Series 3 where one day out of five was flyable. This was yesterday from a hill called Camlo which is a “little flown hill, especially for hang gliders, as there is not a suitable bottom landing field”

Numpty here took off first with everyone watching and waiting because there were no clouds, and I got thrown about a bit. Luckily there were no other gliders in the air to dodge.

Then I came in and landed when things got full up like a zoo (orange glider on the 39th second of the vid).

Then I un-redeemed myself by sitting on takeoff for most of the next video until the launch window was about to close.

Most of the big boys got away during that time until I was one of just two gliders left in the air. I went in and landed just as the other one got away to a big cloud. Grumble.

Once you’ve tasted XC flying, nothing is ever good enough again.

Monday, August 15th, 2016 at 9:20 pm - - Cave

This one has been on our minds since we were caving novices in CUCC (back in 1987) when some of the big boys went to dive the sumps in Rowten Pot and had a bit of an epic. Twenty-nine years later Becka and I used the rigging put in for the Eurospeleo conference, took a small pony cylinder and checked it out.

It was lovely with masks, warm hood, bright LED lights, proper wetsuits (much better than what they had in the bad old days). We went back and forth a couple of times without the tank, and then exited through Valley Entrance — with some complaining at the short pitch from the streamway, which I climbed out and she whinged at, in spite of the fact that she goes bouldering most weeks, and I don’t because I hate it.

Saturday, August 13th, 2016 at 12:23 pm - - Machining

I’ve been getting some machining done, found out what worn out tools are like when you try to use them, and bought a whole load more.

We’ve also been doing some important work controlling servos directly from the Beagleboard through an H-bridge which will allow us to bypass the conventional servo motor drive setup that prevents all positional feedback information.

You can see the varying width modulations on the scope here:


The code, compiled from C and deployed on the programmable realtime unit, varies the pulse width every cycle thus modulates the amplitude like a class D amplifier.

The music, in the second half of the above video, isn’t well picked up by the phone microphone, although it sounded surprisingly good in reality.

This is almost all the PRU code responsible for reading the channels (pairs of numbers that are wavelength and amplitude) and setting the PWM width per cycle.

struct Commondata {  unsigned int notes[NOTECHANNELS*2];  };
volatile near struct Commondata commondata __attribute__((cregister("PRU_DMEM_0_1", near), peripheral));

int main() 
  CT_ECAP.ECCTL2 = 0x02C0;
  CT_ECAP.CAP1 = 4096;    // PWM clock period
  CT_ECAP.ECCTL2 |= 0x0010;

  long channelcounts[NOTECHANNELS];
  for (int i = 0; i < NOTECHANNELS; i++) 
    	channelcounts[i] = 0;

  int mid50 = CT_ECAP.CAP1/2;
  int guardlo = mid50/8;   // clock range safe to change
  int maxamplitude = mid50 - guardlo;


    int waveamp = 0;
    for (i = 0; i < NOTECHANNELS; i++) {
      if ((pid.notes[i*2] == 0) || (pid.notes[i*2+1] == 0))
      channelcounts[i] -= 16;  // granularity of steps per PWM cycle
      if (channelcounts[i] < 0)  // countdown crosses zero, set wave back up
        channelcounts[i] += pid.notes[i*2];

      // calculate sawtooth wave amplitude within period
      int amp = (pid.notes[i*2+1] * channelcounts[i]) / pid.notes[i*2] - pid.notes[i*2+1] / 2;
      waveamp += amp;   // sum up the amplitudes

    // attenuate rather than truncate
    waveamp = waveamp*maxamplitude/(abs(waveamp) + maxamplitude);

    // wait till we are outside of the lower values of the clock
    while (CT_ECAP.TSCTR < guardlo)
    // wait till the clock has ticked over (but hasn't counted above 100)
    while (!(CT_ECAP.TSCTR < guardlo))

    // set the next duty cycle length (safe as value greater than guardlo)
    CT_ECAP.CAP2 = waveamp + mid50;

It's about 50 times less code and effort than I thought it was going to be, mainly due to the use of the PWM subsystems, rather than attempting to generate timed PWM in realtime from the PRU code. This could only have been possible were I authoring the assembly code directly instead of using a compiler, because only that way can you count the cycles. This is a good move because, while it would have been fun to write carefully hand-coded division routines that used a fixed amount of processor cycles, it would have been a waste of my time. This way we're simply using the PRUs as co-processors that we assume are running fast enough to service those other subsystems in the chip.

The other subsystem we need to read are the quadrature encoders in order to drive the motors to a given position -- and read what position they have been dragged to by other forces (like human hands).

This is the current state of the project plan to get to a milling machine that responds to cutting forces with changes in toolpath trajectory (which requires the abolition of G-code).

It's all about us keeping an eye on where we are trying to get to, picking the route to it that takes us across the known unknowns in a way that exposes us to maximum learning, and then cutting corners (eg by further bodging the circuits we already have).

I couldn't do this if I had to explain, repeat and clear every change of plan with a boss who was remote to the project.

On the other hand, if we did have a boss maybe we'd get there at the same rate on account of a lot fewer holidays per year.

Thursday, July 28th, 2016 at 11:25 am - - Flightlogger, Hang-glide 1 Comment »

I don’t know which direction to go now, so I did a quick bit of video editing of a flight in Italy. I got plans to write this story up in more detail if I can get over the writers block that’s making it difficult even to complete my logbook.

I am losing control over direction. I should be running the machine tool on something, to keep it oiled. But then there is the GroundWindow app that I’m converting to work in Yorkshire with the cave data we have there. And also analyzing my flight logger data, which I seem to put hundreds of lines of code into a week, but continues to get nowhere. It’s strange.

The WebGL situation with GroundWindow is diving into a real rabbit hole. I have long known I needed to learn how to code GPUs, but could never find a way in. GPU technology, as I have observed, makes much of the software engineering I have done over the years entirely redundant.

Last week I spotted that the genius who made the early webgl water demo I’ve been hawking around to prove my point finally written a blogpost about one small aspect of the demo.

In it he referred to another blog about the GPU graphics pipeline (written in 13 parts), that I am trying to work through. I’m getting about 50% of it at most.

The question then comes down to whether there are any meaningful new machining algorithms that can be built based on this technology, using OpenCL or whatever, because that’s one of the few things I am uniquely qualified to research better than most — even if I can’t persuade someone with a budget to pay for it.

That’s just normal stupidity and mis-allocation of capital by those in control of developmental budgets.

For example, let’s take the massive public engineering program called Trident just authorized by Parliament this month to build four nuclear powered submarines to provide that essential round-the-clock service of threatening indiscriminate annihilation to targets unknown at a few minutes notice.

Now some MPs believe that investing in high technology is good merely for the sake of it, like Toby Perkins, who said in the debate:

The most depressing exchange was with representatives of the GMB union in Barrow [where the submarines are built], when… [the MP] for Islington South and Finsbury suggested that they might like to make wind turbines instead. They politely but firmly informed her that they were involved in designing and producing one of the most complex pieces of technology on the face of the earth, and that wind turbines had already been invented.

Setting aside the fact that nuclear submarines have already been invented too, the difference is that wind turbines produce electricity, which has value. Military nuclear submarines, on the other hand, have no return on investment. They are not put for up sale as part of the international arms market to foreign dictators (and you won’t get away with selling used ones to Canada again). The subs are not applicable to a modern resource war, like the invasion of Iraq where the plan was to win the wealth back by stealing their oil, because the victims don’t have navies. And there is no program for technology transfer, given that the nuclear power industry has been entirely outsourced to France on a strategic level

In fact all the engineering being budgeted for this program is wasted and will be subtracted from the engineering brains available nationally, just when we need them most and the availability of immigrant engineers is going to be choked off.

Nuclear war, in terms of the way the politicians handle it, is worse than low-grade Science Fiction. So at this time I picked up the 1964 Heinlein post-apocalyptic novel Farnham’s Freehold, where an all-out nuclear war blasted the Goldwater republican right-wing Americans (with the same mind-set as the author) two thousand years into the future from their private fall-out shelter. Here’s one of the characters in the future civilization looking back at the recorded history trying to interpret the events:

The war itself he didn’t find hard to believe. He had experienced only a worm’s-eye view of the first hours, but what the scrolls related matched the possibilities: a missile-and-bomb holocaust that escalated in its first minutes into ‘brilliant first strike’ and ‘massive retaliation’ and smeared cities from Peking to Chicago, Toronto to Smolensk; fire storms that had done ten times the damage the bombs did; nerve gas and other poisons that had picked up where the fire left off; plagues that were incubating when the shocked survivors where picking themselves up and beginning to hope–plagues that were going strong when the fallout was no longer deadly.

Yes, he could believe that. The bright boys had made it possible, and the dull boys they worked for had not only never managed to make the possibility unlikely but had never really believed it when the bright boys delivered what the dull boys ordered.

Not, he reminded himself, that he had believed in ‘Better red than dead’–or believe in it now. The aggression had been one-sided as hell–and he did not regret a megaton of the ‘massive retaliation’. [Chapter 14 p190]

Two things: Being ‘red’ is actually a temporary phenomenon (unlike radioactive and dead). Just ask the East Germans.

Secondly, the Cold War was stoked and prolonged by the dull boys in America, from their endless lies about the missile gap, to their intrusive U2 surveillance flights across Soviet airspace that utterly wrecked the four powers peace summit that had been scheduled to de-escalate the Cold War in 1960.

Ironically, those U2 flights were collecting intelligence that proved there was no missile gap whatsoever, yet the President and Presidential candidates continued to lie about Soviet capabilities to paint their political opponents as “weak on defense” in the forthcoming election.

It’s the old game of elites clinging to power by scaring the bejeezus out of the public, and then offering dangerous answers that don’t work, and successfully displacing consideration of the real problems at hand that require solutions they don’t want anything to do with.

The problem with our thinking is that future exists only in the human mind, and we are not carefully discriminating between the challenges ahead that are entirely within the various states of mind, such as the threat of war and the causes and consequences of property distribution and financial debt– and challenges out there in the physical world that are not going respond to any of our puny beliefs, like climate change and the polluting energy systems in the modern world.

In a sane world the Committee on Climate Change would get the £200billion engineering budget to start building the stuff we need now, like tidal barrages and CCS, and the nuclear warriors would instead sit in smoky committee rooms writing strategic reports on paper and getting sent off to international conferences to sign treaties– in other words do the sorts of things that would solve those problems completely.

That’s the way round it should be. But it’s like we think we’re looking through a window on the future, and instead it’s just a mirror reflecting the past behind us. And this would be fine, if it weren’t for the point of reality that time does not in fact run backwards.