Freesteel

Apparently the following terribly obvious idea isn’t terribly obvious, and everyone who has been collecting time series data from, say, temperature sensors has simply been stuffing them into their flavour-of-the-month database as, basically, rows of the form (time, value, sensor_id) in copious quantities with not much idea about what to do with it.

If you’ve spent your life generating machine tool paths and thinning out the points so that the controllers don’t get overloaded, the first thing you say is: “Hang on, why don’t you just fit some straight line segments to all this data?”

Our good friends at Open Energy Monitor immediately began running with the idea as soon as I mentioned it — and show every sign of comprehending the depth of the science behind the practice which I have unfortunately not had time to investigate in our machine tooling case.

Here is a section from a temperature time sequence that contains 10090 values collected over seven days:

Here’s what it looks like when you thin it to the tolerance of 0.25 degrees which results in 210 points — a compression factor of 50 times:

The important thing about these algorithms is to model the lines, not the points, because that’s what you will be integrating with. This means a record in the database is of the form:

(time0, value0, time1, value1, tolerance, sensor_id)

Now you might think this is unnecessary redundancy because each time and value is listed twice in the database, when you should be able to get away with simply listing the sample points once, but it’s actually the linear segments that define the curve. Databases are very poor at joining rows against sequentially subsequent rows to recover these records, so don’t try to do it that way. The tolerance value refers to how well the line segment from (time0, value0) to (time1, value1) fits the data.

The algorithm I use to do this thinning is pretty simple and recursive on a stack:

segs = [ (t0, v0, t1, v1), (t1, v1, t2, v2), ... ]  # input
thinsegs = [ ]                                      # output

cstack = [ (0, len(segs)-1) ]
while cstack:
    i0, i1 = cstack.pop()
    di, dtol = GreatestDeviation(segs, i0, i1)
        
    if dtol > maxtolerance:
        cstack.append((di+1, i1))
        cstack.append((i0, di))
    else:
        thinsegs.append((segs[i0][0], segs[i0][1], segs[i1][2], segs[i1][3], dtol))

All we do is fit a line to each sequence of segments, find the element of greatest deviation (on the (t1, v1) point), and split at that place if it exceeds the pre-set tolerance.

Doing it this way in a batch is quite a bit easier than attempting to thinning ongoing at load-time, but it is simple-minded because on an evenly curved path it will tend split to power of 2, which means it’s on average around 25% worse than optimal. And sometimes it’s possible to fit lines to the sequence whose endpoints are not on the sequence, but where the lines fit better. I’ve never done this; it’s hard, doesn’t gain much and may have disadvantages. But it is an important part of the science. One thing I do like about this algorithm is it preserves corners well (although there are some exceptions). I also think that the tolerance should be variable, so it can be tighter where the slope is long and shallow because the low frequency undulations on a long stable sequence could contain important data.

A notable observation of this curve is that it looks it contains exponential decay segments going up and down as the person in the room switched on and off a fan heater to keep warm. The decay curves going down are much clearer, and I wonder if they are all tending towards the ambient outside temperature at a rate given by the insulation rating of the room (very poor).

So we tried fitting exponential decay curves to this data, made possible with the addition of a further parameter in the segment record, called ex.

    v = v0 + f (e-(t - t0) ex - 1)

where:
    f = (v1 - v0) / (e-(t1 - t0) ex - 1.0)

This reduced the segment count to 161, which is promising. I can’t plot the curved segments using the annotated timeline, so they show up as straight lines below. However, I can plot the exponent values which show that the decay rate is pretty steady across the intervals whenever the heater is switched off:

So that’s the fun and games we can play with one sensor alone. Imagine what we can do when we have 100 sensors all in the same house and can relate the time displacements of the temperature changes between one sensor and the next.

Half way through June and it feels like I’ve already blown this summer. Got to get busy.

On Saturday the 1st of June in the afternoon, I did a kayak dive on the SS Editor (wrecked in 1887) on tide rip rock at Penrhyn Mawr where the sea kayaks like to play. A bunch of them came through while I was thinking about diving, and I should have filmed them for context. Otherwise, it’s a more watchable vid than most and could be popular with those paddlers curious about what’s on the sea bed.

For me, this dive was a collector’s item. I hadn’t expected to kayak dive such an extreme place, but the psychological boat cover from our club diving the Missouri nearby gave me the impetus to go for it. Odd to note, my quivering fear of these actions seems to have vanished and I just got on with it. I don’t know why.

On Sunday 2nd of June a cave diver friend came out to Anglesey for a dive, having spent all Friday night in Large Pot on a trip where they grabbed 400m of new cave passage without surveying it. This was extremely rude and thoughtless, but we forgave him. Sea conditions were ideal, and we headed over to the wreck of the Hermine. I couldn’t recharge the gopro, so there’s no movie of the dive in which every living sea creature was encountered in its proper place, from the conger in its hole, to the crayfish on a ledge as though in an aquarium. Becka played surface cover in a boring sea kayak. We took a second dive out to 20m from the Hermine (having juggled diving bottles between us) hoping for a drift dive, but there was no current. There were, however, scallops. And that was the weekend, where I got 5 dives and Becka only got 2, but our visitor got 2 as well so it wasn’t completely wasted.

Becka spent the whole of the week agonizing over the extensions in Large Pot and couldn’t wait to get me underground at the weekend on Saturday 8 June when we would survey all the passages the first explorers had trampled through, but not drawn up. Her plan was that we would go down on two consecutive days because there was too much to survey in one trip. I knew this was implausible because (a) the length of the new discoveries was grossly over-estimated, and (b) I become very grumpy when I do more than 6 hours of caving per week — particularly in a very grubby wetsuit that needs to be patched after every trip. The mud in one of the crawls was exactly like Philadelphia cream cheeze — soft and very sticky. You won’t forget it. Hopefully the passage of lots of visitors will eventually smear it out of the way so it becomes a dim distant bad memory.

There’s not a lot of point taking a gopro underground in big passage, even with the brightest light I have. The Olympus camera that would have worked is terminally broken again after being taken on its second trip. This video shows us putting down conservation tape into the fresh passage so that cavers can keep to one well trampled path and avoid trashing 100% of the cave floor with their muddy boots.

On Sunday we stomped around the hillside looking for evidence of our discoveries on the surface (eg where this huge tunnel would have intersected with the valley) but drew a blank. Then stomped up Whernside looking for a place I could potentially fly my glider sometime when there is an east wind and I have energy for an insane carry up.

Finally, on Monday, I did actually fly my glider with almost disastrous style. I’m not pleased with it, so here is a link to the unlisted YouTube video. Fasten your seatbelts. It rapidly turns sheep shaped.

Stunning weather. Astonishing visibility. We had the guts to go beyond the scary Penryn Mawr into the sheltered coves beyond and dive the wreck of the Kyle Firth just beyond.

This wreck is right about here:

View Larger Map
More vids to follow.

Blazingly sunny day over Wether Fell, but with blue sky above the hill all day. About 30 paragliders could be seen rising above Dodd’s Fell up ahead, but there never was quite that kind of lift where I was, with another 20 paragliders on my hill, wafting about the place, catching the once-every-half-hour breezy thermal (which I missed, as I was waiting for it to get better, which it never did. Lesson learned.)

Finally went for it at 2pm after two hours standing under the glider, and went straight down with barely a breath of lift.

There was so little wind it didn’t matter which direction I landed in.

(more…)

Believe it or not, I’ve been working on the Adaptive Clearing strategy recently, attempting to upgrade it to have multi-core capabilities. As you know, the free lunch was declared over 2005 when CPU clock-cycles stopped increasing exponentially. Now, instead, the numbers of cores are increasing. I have 8 cores on my current laptop, and when my state-of-the-art-algorithm runs on it using only 12.5% of the computing capability, it’s embarrassing.

To break the algorithm down into components that can be processed concurrently, I have separated it into 7 working threads, each of which pipes objects to the next layer through a queue. I was told that this is pretty standard architecture — even though it doesn’t look like it can be done using the concurrent.futures module in Python (where I looked for inspiration) because it looked like it needed the complete list of tasks before it could distribute them — at least in an earlier incarnation — and not a pipeline of data. I was also confused by the use of the word “futures” for processes not yet complete, which was similar to the __futures__ module which was features from a later version of Python into an earlier one. I don’t like the terminology anyway, but that’s just my opinion.

Anyway, a Queue is a structure that has functions queue.put(object) [to the front] and queue.get() [returns an object from the back], where each end can be handled by a different operator, like like this:

Let’s say op1 is sending in horizontal slices of the model (the objects obj) and op2 is calculating the clearing toolpath at each of the horizontal layers. We can run these two operations in different threads, and it will all work tickety-boo. The operating characteristics of the threads getting and putting from and into queues is that when a thread gets from a queue that it empty, it automatically sleeps (consuming no processor cycles) until something is put into the queue at the other end, at which point it automatically wakes up and retrieves the object. The opposite happens on the put if you have specified an upper bound for the capacity of the queue and you hit it.

(more…)

I have a confession to make. The digging trip in April wasn’t quite as boring as said.

In fact, we broke through. But since it would be about a month before the digging team could get back together and do all the stuff we needed to get done before the teeming hoards went in, we kept it secret.

To recap, the dig started in the dry mud floor in this terribly obvious place here:

(more…)

This is paddling back along Loch Carron from Plockton to our lovely hostel in Stromeferry with a 20mph tailwind on Monday 13 May.

(more…)

I don’t usually talk about plans like this too far in advance, due to the fear of jinxing it by some bad event, but I am too much looking forward to this, and I don’t have anyone to go with yet.

(more…)

In their new executive-summary-style webpages, the UK Government sets out its case that it is fulfilling its side of the deal that allows them to remain a nuclear weapons state under the Nuclear Non-Proliferation Treaty wherein they promised to “pursue negotiations in good faith on effective measures relating to cessation of the nuclear arms race at an early date and to nuclear disarmament, and on a treaty on general and complete disarmament under strict and effective international control.”
(more…)

The Long Mynd hang gliding site is sort of on the way to LLandrindod Wells where we were going to bury grandfather’s ashes. He once owned a shop in Church Stretton nearby and talked about the gliders up on the Long Mynd in the early 1960s when he was out rambling. So it seemed like a good idea to go there and take advantage of the westerly wind.

Unfortunately at a wind-speed of 20mph, it was a bit breezy, so there were not many gliders there, even though it was a bank holiday. None of the pilots were local; the locals can pick better days when it’s going to be more fun. This was going to be a gale-hang. Two people had flown. Now it was my turn, having delayed until one of them had landed and could tell me the conditions (quite smooth).

Everyone had Wills Wing gliders similar to mine, not the fancy topless kind.

I had a good time — except for the landing. Actually, the landing was fine, it was the 3 seconds before it that weren’t so good. I should consult an instructor to work out what happened and whether the harness configuration needs thinking about.

I don’t know what I’m doing. At least the camera tells me my posture is terrible. You’re supposed to pull your weight sideways to turn the glider, but it looks like my legs go the other way half the time, which means I’m simply twisting and not shifting my weight anywhere. The picture shows me that the VG cord makes a difference to the flex of the wings, though I’m too insensitive to feel it because I can’t even tell the difference between turbulence and crap flying.

Also, this new flight computer doesn’t make much sense. I can plug it into an air speed indicator, and it thought I had already taken off due to the wind speed of 20mph, which was roughly the speed of the glider. When I pulled the bar in it could go up to 30mph.

The top graph shows that I immediately rode the rising air up to 350m above take off, then straight-away bombed back down before anything else happened. This coincided with my brief excursion north along the ridge in the right hand chart. Having lost all this height so quickly, and hated it, I hung out over the take off area for the rest of the flight, not wanting to risk going anywhere else again. A sailplane buzzed up and down the ridge below me. Much later another glider joined me, rising to my altitude in no time. It went along the ridge to the left and then back towards me, and was a lot more interesting to watch than my boring flight, according to Becka.

I want to go somewhere one day so I don’t just have map scribbles. But I don’t have the guts or the skill. I get all these notions when I’m on the ground, but it completely falls apart as soon as I take off. I’m sure there something obvious I’m missing out on. It’s like I’ve learnt the wrong shape of the air.

We stayed in a nice B&B on Saturday night, then in a not so nice hotel on Sunday night with the rest of the folks. Becka headed off early Monday morning for a scary caving trip in Pool Park with its heavy steel padlocked lid over a 100m entrance pitch (she dropped the padlock) on a nasty dry 9mm rope. Too scary for me. They call it a fear of heights, but it’s actually a fear of falling. The height is not a problem if you are flying, just as the depth of water doesn’t matter if you are floating. But if your hands are tied together and you are wearing lead ankle chains, then water gets really scary. I caught a lift to Cambridge for the day, and expect to be on a train home tomorrow night.