skip to main content

Munged.Org Where Wookie waxes loquatious, because he can.

Using awk To Add Lines To Files

I had to stop and figure awk out again. I used to be almost ok with awk, but it’s been a while.

I have a bunch of files from previous incarnations of this blog. I want to add redirects from the old “munged.org/saga” locations to the new Jekyll “2015-03-27-title-of-post” location.

There’s a whole ‘nother decision about maybe just specifying a permalink in the source files to put the resulting html files in the /saga/ directory, so they show up where they used to. All my testing says Jekyll doesn’t have a problem with posts scattered all over the place via permalink.

But I kinda want to “move on” on this one. Plus, this is also how I’d put in the permalink variable in all the files anyway, so this work is not a bad thing.

And, I enjoyed dipping my toes in this again. What the hell, ya know?

My requirements:

  1. I’m going to only add lines to the files
  2. I want to add the new lines just above the second ‘- - -‘
  3. The files must otherwise not be modified

The lines I need to add are:

mtid: 1
redirect_from:
  - /saga/1.html

Where ‘1’ is going to be a record number from the Moveable Type system. I have files with ids as high as 507.

I’m adding the mtid just to keep the ID number around. Right now, it is part of the source file name, but I want the data IN the file for safety.

For awk, I need two rules: one to write all the lines into the result file, and another to add in the new lines before the second ‘- - -‘. The two rules are:

# pass 'id' in via the -v switch on the awk command line
/---/ && NR > 1 \
    {
        printf "mtid: %d\n", id
        printf "redirect_from:\n"
        printf "  - /saga/%d.html\n", id
    }
# print all lines
    { print $0 }

First, we select all lines with three dashes where the record number is greater than 1, so we select the second such record. Then we write the new lines into the file. The variable “id” has to come from the awk command line. Finally, we write the line out to the output.

The order is important. Since I want the new lines before the ‘- - -‘ AND I want the ‘- - -‘ in the output, I have to trigger on the ‘- - -‘ and add the new lines before “print $0” writes the ‘- - -‘ line out.

This is invoked like so:

awk -v id=504 -f ~/addid.awk source-file.md > output-file.md

If I was bulletproofing this, I would need to do two things that I’m not worrying about on this pass:

  1. Make sure the ‘- - -‘ I’m getting isn’t part of a markdown “heading 2”.

    One of the ways of creating a <h2> heading is:

     My 2nd-Level Heading
     --------------------
    

    I don’t have any of this right now, so I’m not going to fix this.

  2. Put in a BEGIN rule that checks to make sure ‘id’ is passed in from the command line.

I’ll put the scripts in the blog repo if you want to look at them.

Historical Note about Munged.Org

My blogging activities started shortly after the terrorist attacks of September 11th, 2001 happened. On November 12th American Airlines flight 587 crashed on takeoff from JFK. I was in London for work that week, and my voicemail pretty quickly loaded up with family asking where I was and if I was okay. I was, but I didn’t have a mobile phone that worked in the UK, so I had no way to listen to them, or respond.

I decided I needed a way to let my family know where I was, so they could have some idea of the chances of me being in some major catastrophe. The US was kind of in the mindset at the time that catastrophes were something that were going to be a fact of life from then on. Planning on and around them didn’t seem like an odd thing to be doing.

The sensible thing, looking back, would have been to use a shared calendar somewhere on the web, and give my family access. I have no idea why that didn’t seem like a good idea 13 years ago.

Instead, I started posting on a website my comings, goings, and whereabouts, in a “web journal” format. The site was static; I wrote the first html pages by hand using the vi text editor.

After editing by hand for a while, I moved to using Dreamweaver. The site was still static, but Dreamweaver allowed me to edit in WYSIWYG, and send the resulting files to the web server with the push of a button. Even that’s a pain tho, so I moved from there to using Moveable Type. Munged.Org was based on MT until about 2009 or so. I made almost 500 posts on that system before I joined Facebook.

I joined Facebook in 2008, and started actually participating in 2009, and that killed any blogging I might have continued doing.

I hadn’t been writing in the blog much anyway, because I was travelling so much. My time all went to dealing with trying to live a life in hotels and airports. But while the shiny lasted, FB was a lot of fun, and my friends turned out to be interesting, creative, hilarious, people.

The idea of writing a blog wouldn’t die tho. Sometime since 2009, I set up a WordPress blog on the same web server I’d had the Moveable Type blog on. I didn’t post much. Those posts I did make are gone, because when I finally took that site down, it doesn’t seem I managed to save anything.

I also tried Tumblr for a while. Again, not much went up there, and none of it saved when I stopped using it.

I even upgraded Dreamweaver again at some point, and was going to do a static web site again. I never got that attempt to the point of having it show up on the web.

Last year, in April of 2014, to be precise, I created a web site with Squarespace. Munged.Org is there as I type this. Hasn’t been much going on over there, tho.

And now you are caught up. Almost.

Going Native or Going Somewhere Else

My Squarespace subscription is up towards the end of April. So I’m playing around with it to see if I want to stick around here. The new template is called “Native”. It is cleaner than the old “Bedford” template, but still obviously way more focused on the visuals than I feel Munged.Org needs to be. Pictures can go up over on WookPhoto.

Some things that Squarespace made easy: The gallery of DC25 photos was pretty easy to put up and looked good. Adding any pictures has been pretty easy, and includes basic photo editing controls to make them look like I want. Easy integration to YouTube and Vimeo videos for the Phantom. But it isn’t perfect. The link tool will show some of the content of this site to link to, but not blog pages. I had to save this as draft, go find the link to the DC25 gallery, then edit this file again to paste it in. Hope it works the way I formatting the link. Edit: Nope!

Where else would I go?

I’m currently testing moving the site over to GitHub Pages. It also has some interesting implications about security and accessibility that may be considered “features”, or not. For one thing, because I’m not paying for Github, the source of the site is in a public repository. Everything is visible: page templates and formatting code, plus the content source.

This means that all y’all could clone the repo, fix things, and send me pull requests to include your fixes on my site. What would you change? Spelling. Formatting. Site changes. Fact checking. Adding interesting links. Whatever you are willing to spend the time to do to make the site better. It is my option to include any changes, but it is an interesting idea.

There’s also an issue tracker I’m using to remind myself of things I want to do, and a wiki that can be used for off-site collaboration. The wiki is intended for documentation, and I have to admit the idea of my blog having “documentation” amuses me.

The downside is that it isn’t really easier than Squarespace. The repository has to be cloned to my computers, and edited locally. Changes are version controlled, which is nice, but live locally until synced with the main repo on GitHub. If I forget to sync, I could take the laptop and not have the most recent changes that live on the desk machine.

I’m using GitHub and git, for the first time, so still kind of feeling my way along. I GPG signed a tag (version release) last night at a completely inappropriate spot in the development of the site for example. Tho really, that was just to see what it looked like. I might, once I get everything the way I want it, blow away the repo and re-create it from files saved locally.

If I move over there, I’ll make sure to import all the posts from here as well. There aren’t that many posts here, it will be easy. I might, if I end up with the energy and interest, import the really old site from the Moveable Type days. Sadly, the WordPress site is gone. But there wasn’t much there anyway.

It is very interesting, but will kind of be a lot of work for nothing if I stay here. Them’s the breaks, I guess. Or maybe I’ll give Textpattern a try again.

New iPhone and iPad

The plan is to get upgraded to current iPhone, and iPad. I have a new iPhone 6, 128 Gig on Verizon. Those phones are hard to find out in the wild, but for assorted reasons I couldn’t order it from the Apple Web Store. I found one in Albuquerque last week. I did order an engraved iPad Air 2 from the Apple Web Store, and it was waiting for me when I got home yesterday.

I just need to get Ally an iPhone, and then the current upgrade cycle is complete. I should actually be able to get her phone thru the Web Store.

Ally was already on Verizon, and I have switched to it. I have mixed feelings about this. It is really nice to be able to use my personal phone in my house. AT&T didn’t serve this area very well, and I had a lot of dropped calls. My work mobile phone is on Verizon, because I had this same problem in DUMBO before I moved to the woods of Carrboro.

Verizon has good coverage, and they use CDMA, so the call quality will be good. But they are just evil incarnate. I switched from Sprint to Verizon back in the day, so I could get a Motorola Razer flip phone. It would access the internet, and I even tried it once. There was a “landing page” that indicated if you proceeded, you’d be using the internet and would be charged for it. What that landing page didn’t say was that it itself was on the internet, and you were already being charged for using it. This sleazy move got Verizon slapped with a lawsuit, which they lost.

So. The ability to use my phone from home if I so choose. Or not supporting evil asshole corporations. Kind of an ugly choice.

After a few years of shitty “service” on AT&T’s part, I am done. I am on Verizon. I could have gone with Sprint, but I had an iPad Air 2 that I returned, and had it on Sprint for a while, and didn’t have good coverage from home with them either.

I still feel queasy about this choice tho.

Categories

I can assign categories to posts. Problem is, I’d rather have one set of categories, and use them forever more, with the occasional addition as new things come up.

Or, I’d rather have a small set where each is broad enough that I won’t ever need any new ones.

Or….

Sigh. Crap.

There are like 4 posts on this blog, and already I am running into things that should only bother me when I realize I have 200+ posts in the category “Opinion” and another couple hundred in “Pontification”. Both categories being the same thing in my mind.

This is partially a case of having a feature and being determined to use it no matter how useful it is. As a tech support goon, I see the CONSTANTLY in other people. You’d think after seeing the evil it causes, I’d be immune. But I like my toys too, which is basically what a feature is.

So, ideas.

  • “Show”, “Tell”
  • “YMMV”, “IANAL”, “ROFL”, “OMW”, “IIRC”, etc
  • “<A>”, “<BLOCKQUOTE>”, “<LI>”, etc
  • “Car”, “Life”, “Event”, “Opinion”, “Site”

Right now, as I write this, I’m leaning towards the internet slang abbreviations. Damn Brett Glass for taking ymmv.com back in the 90’s and not updating the content on it in over 16 years. That’s just mean.

Disqus

I’m trying out the Disqus comment platform, since SquareSpace integrates with it natively. Leave a comment, so we can see how it works.

Update: No, we’re not. Doesn’t seem to work very well, and seemed kinda complicated when it does work, so we’ll just go with the default Squarespace comment system.

New Tires for Wook 1 - Sort of a Review

A couple of week ago I bought new tires for Wook 1, to replace the Cooper Zeons it had. I got Michelin PilotSport A/S 3s. It is coming up on winter, and there will be at least one trip up North during a period of time where there could be snow on the ground. So, all season tires. I drove around Des Moines for years in cars with half-bald normal tires with no problems. Full-on winter tires just don’t seem required.

That’s not because Wook 1 is 4WD, because that’s not the reason. 4WD helps you go, but tires make you stop. It is just that I’ve never driven a car that had snow tires on it, even when I lived where it would have made sense to do so. If you are careful, which I am, it isn’t a big deal.

When you go on Tirerack.com, there is no “normal tire” in the 255/35ZR19 size Wook 1 likes. You either get some kind of summer performance, winter performance, ultimate performance all-season, or even more aggressive type descriptions. So I went with the “Ultra High Performance All-Season” category, and got the PS A/S set.

Summer max performance tires, like the PS3s or P Zeros, are Right Out. At least right now before winter. I know this from when I first bought the car, and it has Pirelli P Zero Rosso max performance summer tires. I was in Des Moines, it snowed, and I did a full-speed power slide thru a busy intersection. Sideways. Next morning I took it out to McDonalds and it was like trying to drive and stop a toboggan on a steep hill. Next day, it had the Cooper A/S tires on it, and it was very sure footed.

But I hated the Zeon’s. They were loud. They had a weird road noise that inside the car often sounded like metallic grinding. This had me really worried until I figured out what was going on. When they sat for a day or two, they would get a flat spot and would be bumpy on smooth roads for the first 10 miles or so. That was added to the fact that their ride was just rougher than the Pirelli’s to start with. In their favor, the Zeon’s did have really nice traction on ice and snow. That, they did do well.

I had to replace them tho. I hit a granite curb in a parking lot, and not only dinged up the rim, but took a chunk out of the sidewall of the tire too. That damaged tire was on borrowed time from then on. I am astounded that it lasted another 800 miles until I could replace it. But that was reason enough to replace that one, and I replaced all 4 all the way around.

So far, the Michelin PS A/S tires are nice. They are quieter, in my experience, tho not as quiet as the Pirelli’s were. They ride nicer on all surfaces. The only real complaint so far is that they seem to be overly squirrelly on grooved highway surfaces. Even the lightest grooves seem to bring this out in them. Ally noticed it from the passenger seat. I’m hoping that a thousand miles or two will even them out a little. The original Pirelli tires tramlined really badly for the first 2000 or so miles, but then settled down. Hopefully the Michelins will settle down after a while too.

I’ll want to get max performance summer tires back on the car at some point. That’s really when it is a complete pleasure to drive. My plan has always been to buy a second set of rims with the “other” tires on them, so I can jack the car up in the driveway and change them out for the season change. But that hasn’t happened yet. Maybe someday soon.

Can I Borrow A Cat

Today, I was thinking “I should tape bacon to a cat.” I know, I know, it’s been done. But sometimes, you have to do the things you have to do.

But I don’t own a cat, and Ally won’t let me tape bacon to one of hers.

So I guess I’ll have to mark today down as a failure.

Targeted Ads

Check out this piece: Why Do Contextual Ads Fail?

My first thought at looking at the title of this, was “who’s still browsing the internet without an ad-blocker?” (Ad blocking software is another topic for another article here. Someday I’ll write it.)

I’ve always wondered why ads supposedly targeted at my interests and needs never try to sell me something I’m interested in or need.

Based on my experience using Facebook without ad blocking software for a while, I have to assume they think every single guy in existence needs a membership to a dating site. I didn’t at the time. Those ads went away instantly when I removed my relationship status.

The best story I heard about targeted ads was when Bobcat Goldthwait said on NPR’s, “Wait Wait, Don’t Tell Me”:

I was writing a script that had a lot of guns in it and I spent a lot of time looking at guns online. And eventually, I’m not kidding, ads for Arkansas came up.

This is either the biggest fail I know of, or somehow the biggest success. I can’t decide which.

Back to the article tho, one thing that caught my attention was the following:

“You’re the product.” I’ll admit that I’ve used that line myself. But I’ve come to believe that it’s pretty clearly a misleading and unsophisticated view.

With contextual advertising, you’re not the product. Advertisers don’t own you. They usually don’t even get to know who you are. The companies selling the advertising theoretically (and algorithmically) display ads to you if you meet the advertiser’s criteria.

I don’t buy the idea that we’re not the product social networks sell. They aren’t selling information they have on us, they are selling access. Just because my info is anonymous, doesn’t mean the social network could exist without that info. Which still makes us the product.

(First) Flight Of The Phantom

Two months ago I bought a drone. More accurately, an Umannned Aerial Vehicle (UAV). In this case, it is a DJI Phantom 2 Video+ quadcopter. This thing is about the size of a dinner plate without the propellers, it has a built-in camera on a stabilizing gimbal capable of 14 Megapixel stills, and 1080p/30 video. The stills come out in Adobe’s DNG RAW format, which I thought was interesting.

The camera is good, tho not great by any stretch of the imagination. DJI sells a Phantom with a different gimbal designed to take a stand-alone GoPro camera. I didn’t get that one, because I had questions about some of the details and the cost with the GoPro was higher by a few hundred. I was concerned about what it took to get the GoPro to work in a “first person video” (FPV) mode where you fly the ‘copter by watching the video coming from it. I suspect that sooner or later, I will change the Vision+ camera and gimbal out for the GoPro version.

Last weekend I flew the thing for the first time. I have two batteries, each good for about 20 minutes of flight time, and I used them both up. I remembered to turn on the video for some of both of those flights. Over the last week, I managed to cut those videos into clips and put those clips into a movie.

Everything about this video was a first: First flight of my first UAV. First attempts at using iMovie and Final Cut Pro. First time I’ve ever had a YouTube account, and first video uploaded to it. The results aren’t great, but they certainly aren’t something I could have gotten any other way. Next time, I’ll pay attention to the camera settings a little more. This time I just wanted to get it flying. I think there is a not-so-wide-angle setting, and I might see if I can lower the exposure, which bordered on overexposed at times. There is also some video distortion in the form of noise bands when the camera was facing towards the sun. There are reports that the camera is way better if you can get a 2x neutral density filter on it. That’s on the list of things to do.

Apple’s iMovie turned out to be not even sorta intuitive. I struggled with it for 2 days until a friend suggested I try Final Cut Pro X trial version, and see if that worked any better. It did, a little. Both enforce this “event” idea that makes no sense to me at all. I just wanted to see the two videos I had, be able to dice them up into clips, and then assemble the clips into a movie. None of this worked like I would have thought in either product. I’ll check to see if there’s an Adobe Premier trial version.

Phone Thoughts

My iPhone 4S is still going strong, tho it could probably use a new battery. I’ll probably be buying an iPhone 6, probably the small one, because I want an iPhone with the motion co-processor. I use location services enough that I have it on all the time, and GPS just sucks the life out of the battery on an iPhone 4S. The M8 chip’s primary purpose is not to provide GPS tracking, it is to provide GPS with a minimal power drain. The iPhone 5 has this too, but I couldn’t justify upgrading that soon after I got my 4S.

I went from the iPhone 3G to the iPhone 4S, which is a 3 model upgrade (3G, 3Gs, 4, 4S). From the 4S to the 6 is also three models. My contract is way past time where I can get a new phone.

Really, the only decision to make on getting a new phone is not “should I”, but “what carrier to get”.

AT&T sucks pretty much anywhere. But at the same time, they also mostly work almost everywhere. The only place I can get all 5 bars of service anywhere near me is in movie theaters. I almost get 1 bar at home. Sometimes.

Verizon has good service all over. But not on Cape Cod at my friend’s house there, so they aren’t infallible. The main problem with Verizon is that they are just pure evil. I had a Motorola Razor phone on Verizon, and the internet landing page that asked if you were sure you wanted to go the internet because you’d be charged for it, was ON THE INTERNET AND YOU WERE ALREADY BEING CHARGED FOR IT. Verizon started the bullshit of always announcing how to use voice mail because it makes everyone use extra airtime listening to it.

Sprint’s coverage is good in some areas. Not so good in others. They appear to have the most data for the least money.

T-Mobile isn’t an option.

I don’t know that I’m ready to get an unlocked iPhone 6 and take it to a small MVNO that uses AT&T or Verizon’s networks. The phone is not cheap when not subsidized, I doubt the money saved by going small would offset the up-front cost of the unlocked phone.

Apple Event on Sept 2014

So.

No new iPad. I’m bummed. I need to replace mine so I have one that works with my Phantom.

iPhone

The iPhone has been a mature platform for years. Improvements to it are strictly evolutionary now, not revolutionary. So we have the iPhone 6 and iPhone 6 Plus. Bigger screens. Faster. Better camera. Better battery life. Yadda, yadda, yadda, yadda.

No news there worth remembering. I’ll probably be getting one, because I want the motion co-processor so GPS stops sucking the life from the battery.

AppleWatch.

Looks cool. I’m sure it will be fun. I might get one. I might not. Ally is all hot to have one for the health stuff, and I might even get it for her. I’ve needed a watch for a long time. So maybe. We shall see.

ApplePay.

NFC is only about the short-range communication between two devices. Two phones, a phone and a computer, a phone and an NFC receiver, whatever. NFC can be used for anything. Payments. Opening hotel room doors. Sending files or contacts. Whatever you can think of. Samsung uses it to transfer assorted data between phones in a nifty sort of way that iPhones accomplish with BlueTooth and WiFi.

In order to make ApplePay, Apple started with NFC, but had to implement everything from there thru the retailer to the banks at the back end. Security, the token implementation, encryption, all of it. This stuff all exists already, but until now it has never been implemented in the USA on the scale Apple seems to have managed.

Apple got agreements from retailers to put the NFC readers in their stores so there would be places for people to use ApplePay. That right there might be the single most important accomplishment of all of ApplePay. That’s what adoption looks like for a payment system: retailers willing to use it. I’d challenge you to name another company that could pull that off.

Systems like ApplePay exist in other parts of the world already. Japan and Taiwan (and probably others) have had this kind ofsystem for years. Those systems are driven by government bodies that set it all up. In the USA, Google and Microsoft have both started contact-less payment systems on their devices. The Google system has been around since 2009 or so. But have you heard of it, or used it anywhere in the USA? It might be unfair, but it seems like only Apple has the mind share to legitimize this kind of thing in this country.

Going forward, it will be interesting to see how it evolves.

And how will ApplePay be fit in to places where something similar already exists, like Taiwan and Japan? My guess is the iPhone will just support the existing systems, but who knows, there might be room for improving the user experience of them.