Forced Static Analysis and charAt

I find forced static analysis tools annoying.  Don't get me wrong.  When I use static analysis tools, I try as much as possible to adhere to those rules (even when I don't agree with some of them).  What irritates me is when the "quick fix" for a rule is not "functionally identical".  Two lines of code are "functionally identical" when you can interchange the two lines without ANY negative consequences.  Static analysis tools like JTest and Sonar sometimes provide a "functionally identical" replacement for a violation.  Someones, the quick fix isn't functionally identical and can cause bugs.  I find this funny and frustrating at the same time, since the supposed point of static analysis tools is automatically find bugs.

The latest one that has annoyed me is the fact that you aren't supposed to call String.startsWith() with a string that has a length of one.  This violation is a performance improvement.  There is an alternative call that is NOT functionally identical that in theory performs a lot faster.  The quick fix will change the startsWith("A") call with a charAt(0)=='A' call.  Here is the theory behind this: startsWith() assumes any length, so it contains a loop.  charAt() has a much simpler implementation.  It just returns the char at the offset + index from the internal character array.  This is not only much faster in of itself, it is also be inlined.  This can be pretty important if you are parsing through a large log file searching for a log line that starts with a given character.  It works fine....until you have a log line that is EMPTY!  That is right; the quick fix for this violation actually causes a bug!

This is where the danger lies with static analysis tools.  It is not about the tools themselves.  The problem with the tools is non-technical people get overzealous with the rules.  A rule may sound good in theory, but it might not be.  It gets worse when you "force" the tools by doing something stupid like failing a build because of a violation.  In those scenarios, you end up encouraging the developers to use the "Quick Fix" features of the static analysis tools.  You end up promoting bugs!

ZFS and dynamic sized volumes

ZFS has an interesting way of handing volume sizes that took me a bit to figure out.  In a normal volume management system, you create a virtual block device (volume) of a fixed size.  You can create multiple volumes, but the total disk size of all the volumes cannot exceed the total pool size.  In ZFS, things are a bit different.  Since volume management is built into ZFS, a volume can have a variable size.  For example, lets create am imaginary 2GB pool in ZFS and LVM.  Now, create a volume in each called "photos".  In LVM, you have to give it a size of 1GB, then format it as EXT3.  Now you add 250MB of pictures onto the volume.  Next, you want to create a volume for "videos".  In the LVM world, you can only create a 1GB volume.  If you want more space, you have to grow your pool first.  In the ZFS world, the "photos" volume wasn't created with a fixed size (unless you accidentally create it in legacy mode like I did).  The df command will report that the "photos" volume started out with 2GB free.  Once you add the 250MB of photos, then it would be reported as 1.75GB free.  Once you create the "videos" volume, df will report it as having 1.75GB free as well.  This means you have the full 2GB of disk space available for files.  There is no dead space at the end of volumes!

Thoughts on Java 8

A buddy of mine asked about my opinion of Java 8, now that the spec has been finalized.  I can't help but feel that Oracle dropped the ball pretty hard on this release.  If you look at the list of new features in Java 7 and compare that to Java 8, you will notice a distinct lack of new features in Java 8.  Here are a few that I wanted to comment on:

JSR 310: Date and Time API

I understand some people's frustrations with Java's current Date and Time API.  This is a welcomed change, but I personally don't mind the current API.

JSR 308: Annotations on Java Types

I have never been happy with Java annotations.  I always felt they could have been done a lot better.  Although JSR 308 does add some much needed annotations (@ReadOnly, @Immutable), I feel like it still fails to satisfy.  In my opinion, annotations should help provide Class invariance, but Java annotations were never good enough to provide that really well.  Also, I still feel that annotations should have code-generation abilities.  I should be able to annotate a class member variable, and the getters/setters should be generated at compile time.  The new annotations still don't deliver.

JSR 335: Lambda functions

Lambda functions can be very helpful.  I am excited to finally have them, but the implementation seems reminiscent of Python's implementation, which I didn't like.  Python seems one step better, though, since it supports function pointers, where Java still doesn't support function pointers.  Maybe its because I can't find any good examples, but Java's implementation doesn't look like it can support complex lambda expressions/functions.

JSR 223: Project Nashorn

I am excited about this.  If I understand it correctly, this is a new Javascript engine that makes use of new JVM features that were introduced in Java 7.  This should allow Javascript execution to be faster while using less memory (especially in the code cache).  I have done a lot with Rhino, so I'm happy about this change.

The Great Bitcoin Arms Race

I got my first back deposit for mining bitcoins.  My wife didn't fully understand what I was doing, so I pulled up a Khan Academy video that explains what bitcoints are.  During that video, I learned something new about bitcoin mining.  I (like many other people) was under the assumption that the faster you can compute hashes, the faster you mine bitcoins.  This turns out not to be true.  The truth is, the faster you can compute hashes relative to everyone else that is mining bitcoins, the faster you mine bitcoins.  This turns out to be a huge distinction.  This distinction creates an atmosphere where miners must constantly invest in mining hardware just to maintain their mining rate.  Building mining rigs becomes an arms race against all other miners.

Bitcoin blocks are mined at a rate of around 2016 blocks every two weeks.  If it takes less than two weeks to mine the last 2016 blocks, then all the miners in the world "decide" to make the mining algorithm harder to solve.  They do this until the mining rate is back up to 2016 blocks every two weeks.  This forces the mining rate to stay roughly constant.  Any improvements in hardware only give the miner a temporary advantage until everyone else has an opportunity to catch up.

Below is a table of awards 4 fictional people would get over a period of 56 weeks.  In this scenario, Bob is very eager to earn bitcoins.  He has money to invest into his mining rig, so he is constantly buying better hardware to increase his share.  Mary is an active miner, but doesn't go out and buy the latest hardware right away.  Alice and Steve are casual miners who keep up with mining, but don't go out and buy hardware until they have to.

Period		Bob			Mary			Alice			Steve
	Type	Rate	Award	Type	Rate	Award	Type	Rate	Award	Type	Rate	Award
1	CPU	10	504	CPU	10	504	CPU	10	504	CPU	10	504
2	GPU	100	1551	CPU	10	155	CPU	10	155	CPU	10	155
3	GPU	100	916	GPU	100	916	CPU	10	92	CPU	10	92
4	GPU	100	650	GPU	100	650	GPU	100	650	CPU	10	65
5	GPU	100	504	GPU	100	504	GPU	100	504	GPU	100	504
6	FPGA	250	916	GPU	100	367	GPU	100	367	GPU	100	367
7	FPGA	250	720	FPGA	250	720	GPU	100	288	GPU	100	288
8	FPGA	250	593	FPGA	250	593	FPGA	250	593	GPU	100	237
9	FPGA	250	504	FPGA	250	504	FPGA	250	504	FPGA	250	504
10	ASIC	10000	1875	FPGA	250	47	FPGA	250	47	FPGA	250	47
11	ASIC	10000	983	ASIC	10000	983	FPGA	250	25	FPGA	250	25
12	ASIC	10000	666	ASIC	10000	666	ASIC	10000	666	FPGA	250	17
13	ASIC	10000	504	ASIC	10000	504	ASIC	10000	504	ASIC	10000	504

Below is a chart of what each person hot as a reward for their effort.

In the chart, all 4 people start out with equal shares of the award.  Bob decides to buy a GPU because he wanted to get ahead of the other 3 people.  His reward shoots up and everyone else's reward shrinks.  Mary notices this and decides to buy a GPU to keep up with Bob.  By week 6, Mary and Bob are making the same award, which is far larger than Alice and Steve's awards.  Alice then Steve each upgrade to using GPUs to keep up with Bob and Mary.  By week 10, everyone is making the same exact award as week 2!  Bob doesn't like this.  He remembers the "golden days" of mining during week 4.  He does some research and buys an FPGA that gives him a little bit of an edge.  Mary, Alice and Steve notice that their awards have dropped a bit, so they decide they have to invest in FPGA's to stay competitive.  By week 18, all 4 are making the same award as week 2.  Bob is really upset at this.  He has invest lost of money (to make money) and finds another upgrade to buy.  He decides to buy an ASIC device and his award jumps sharply.  Just like the 2 previous upgrades, it doesn't take long before everyone else has ASICs and everyone is making the same award again.

In this scenario, Bob keeps escalating due to the desire to get a larger award.  He has money to invest, so he keeps buying faster and faster hardware.  This leaves the other miners in a situation where they either have to invest in faster hardware, or they give up.  In a much larger population, there will be people who give up.  A lot of people will continue to mine, however.  Some may even respond by leap-frogging Bob.

If all the miners agreed to limit the hardware that they purchased, they would make more money.  In the long run, each miner would receive the same award, but they wouldn't have the hardware or power costs associated with the escalation.  This is a great example of the Prisoner's dilemma, however.  Each miner has an incentive to cheat.  The cheating only gives them a temporary advantage, but it is still an advantage.  This is actually studied pretty heavily in economics.  Every miner is essentially part of one big cartel.

In the end, do you participate in the arms race or do you give up?

Why does Paypal still exist?

Paypal has had it's fair share of problems.  They have been accused of stealing people's money.  They are often linked to scams.  I haven't needed a Paypal account in over decade.  They is why I was surprised when I tried to sign up over at www.kiva.org.  For those of you who don't know, Kiva is a non-profit organization for making micro-loans.  A micro-loan is a loan to a business man or woman in another country.  The country is usually a place where the exchange rate means $25 USD (the minimum you can loan) goes a lot further than it does in the United States.  A few people get together and loan the money, then the business person uses it to make their business bigger/better.  They pay back the money, and you re-lend the money.  You end up constantly lending money and helping other countries grow their economies!  What a great idea!  You know what isn't a great idea?  Using Paypal as your money processing system.  Why is it a bad idea?  Because when I try to do a noble thing, like lend money, I get this:

1 Year of Blogging

Last week was the one year anniversary of my blog.  I started this blog to practice my writing skills, and to force myself to research technology.  It also gave me an outlet to vent about problems I have with some technology.  You would be surprised how writing a blog entry can help collect your own thoughts.  When I started the blog, I didn't know how long I would last.  I have found that during my day, I come up with lots of ideas that are worth blogging about.  I also come up with lots of ideas that I thought were good ideas.  My Draft folder is filled with articles that have empty bodies.  I to continue to blog about technology and to put my ideas out there.

Digital Photo Frame Aspect Ratio

I have been in the market for a new digital photo frame recently.  While searching on Amazon, I found some really great looking frames, but they all suffered from the same problem.  They were widescreen photo frames.  This irritates me.  Most cameras take 4:3 pictures, not 16:9 pictures.  This is true for both the analog and digital world.  If 99% of all photos are 4:3, why are there so many 16:9 photo frames?  This doesn't make sense.  A 16:9 frame will display 99% of your photos with either black bars or stretched.  What is the point of that?

Instant Photo Upload

I have been using my Eye-Fi Po X2 with pretty good success.  After coming come from a day of taking pictures, I could turn my camera on and it would upload the pictures I took.  The photos would be ftp'ed to my Linux server directly to a folder that was shared on Dropbox.  I decided to do an experiment to take the being home part out of the process.

I figured up an app on my phone to turn it into a portable hotspot.  I gave it the same WiFi network name and password as my home network.  When, I took my wife and kid to the zoo.  While at the zoo, I turned on the tethering.  As I took pictures, my phone uploaded them to my ftp server which sent them to Dropbox.  This worked for a while, then stopped.  That is when the app told me that if I wanted to continue, that I had to buy the paid version of the app.  I guess they set a daily limit on data.  Since it was a one-time fee of $0.99, I begrudgingly paid for the app.  Then I was up and running again.

After some more time taking pictures, I checked the phone and the uploads stopped working again.  It turns out that after a period of time with no WiFi clients connected to the hotspot, the hotspot turns off.  This is probably a good battery saving feature.  I wasn't keeping my camera on all the time, so the hotspot would turn off from time to time.  This was a bit annoying.

Overall, this proof of concept worked.  It wasn't a perfect success, but it did work.  The end goal is to allow some family members to see the pictures as we take them.  Some of our family is far away and they can't easily join us on a trip to the zoo.  This will help them feel more included with some of the outings that we do.

Playing with ZFS

With some of the recent chaos I've been having with my hard disks, I decided to revisit ZFS for Linux.  The last time I looked into ZFS, it was built using FUSE.  I didn't like the idea of such a complicated filesystem being implemented inside of FUSE.  I have written FUSE filesystems, so I knew some of the more interesting limitations.  This is no longer the case, though.  ZFS now comes as a kernel module for Linux.

After playing with ZFS for a few weeks, I have decided that I love it.  It is definitely missing some features (like the ability to shrink a pool/remove a device from a pool), but overall it is great.  It also has one of the most important features any filesystem can have.  You can force a scan to verify the data on your hard disks.  I have a weekly job that performs the scan, then emails me the results!  No more silent corruption.

I will start posting some articles talking about the interesting and awkward features of ZFS.

Android Compatibility Layer For Linux

Why isn't there an Android compatibility layer for Linux?  I'm thinking of something similar to Wine (Wine Is Not an Emulator).

The reason I want this is because I want a Laptop/Tablet combo.  I want something that runs a full OS.  It should run the full version of Ubuntu.  I should be able to close the lid or rotate the lid or do something to make it look like a tablet.  From there, Ubuntu should let go of the screen, and I should have a full Android install.  What would it take to get that to work?  I'm not talking about running Android in a VM, either.  I'm more talking about Ubuntu for Android, but in reverse.

Android runs on a Linux-based kernel, but it has been modified.  Most of the modifications have been merged back into the trunk, however.  What else is missing?  Control of the video card is an issue.  Only one "think" can control the GPU at a time.  In the Linux world, that "thing" is X Windows.  Every other program talks to X to talk to the video card.  I'm not as familiar with the under the hood parts of Android.  My understanding is that Android doesn't use X; there is some other layer that controls the video card.  For what I want, I think its ok that only one "OS" gets control of the video card at a time.  If X can "release" the hardware so that Android can "grab" the hardware, that would work.  This could be triggered by the keyboard or via a physical action like closing the lid.

Power consumption is an issue.  The Android kernel modifications that haven't been merged are specific to reducing the power consumption of the OS.  In sleep mode, Android will last a lot longer on battery than a regular Linux kernel.  For this effort, I feel like that bridge can be crossed, though.

Networking can get a bit weird.  In theory, you would want to have control over your Wireless network from Android and Linux.  This goes back to the only one "thing" can control a device at a time discussion.  If you won't allow network config from Android, then you can just use a virtual ethernet device using the tap module.  Alternatively, you could write a custom app that communicates with Linux to control the network config.  This is awkward, but would be functional.  This is one area that I don't know how to easily solve.  Optimally, you could create a virtual wireless card that runs in Linux userland.  Any "operations" like listing networks or connecting to networks would just translate to userland calls to the real hardware.  This would probably be a large effort, though.  The virtual networking would allow Linux to create firewall rules for Android, which would be a huge win for security.  Other devices might have this sharing problem.  The keyboard and mouse could probably be handled easily, but cameras probably would take effort.  Audio would probably need to be handled the same way video is handled, unless Android can use the same subsystem (ALSA) as Linux.

Some things are easily solved or are not even issues.  Contrary to popular believe, you can run Android on x86.  Any games that make use of the ARM NDK won't work, but Google discourages the use of the NDK specifically for this reason.  Things like the 'SD' card could easily be virtualized using the loopback module.

Ideally, I would want this Android compatibility layer, or Linux/Android Hybrid on a device like the ASUS Taichi 21.  When the laptop is open, its Ubuntu, but when the laptop is closed, its Android.  I don't think there needs to be any hardware changes to support this.  I feel like this is 100% software, and that software is 100% open source.  This seems like something that can actually be achieved.  The big question is, why hasn't anyone started doing this yet?

CGP Grey....We Want More Videos

One of the big problems with Youtube shows are the lack of content.  By this I mean the low quantity (albeit high quality) of videos per channel.  For example, CGP Grey makes the BEST (scientifically proven) videos on the internet.  Each one of his videos is only a few minutes long and he doesn't release very many, although Subbable might allow him to release more frequently.  We can't ask CGP Grey to make more videos because a) there isn't enough coffee in the world for that and b) that can degrade the quality of the videos.  This is the double-edged sword of Youtube shows.  Some shows have been able to crank out long episodes (like Tabletop) but those shows tend to have a different format that allows them to do that.

This lack of quantity doesn't fill time very well.  There is an argument to be made about not being couch potatoes, but frankly, I like to spend time being educated by other people.  When the shows you watch have shorter episodes, you have to watch more of them.  My wife complains that of the hour or two we spend watching TV every day, we spend 1/2 the time picking the next episode to watch.  Although I have the patients for that, my wife really doesn't.  So, if I'm not sitting with her, she will usually watch shows on Netflix or Hulu Plus instead of Youtube.  This is a problem that needs to be solved if Youtube really wants to tackle Big Media TV.

The solution seems pretty clear though.  CGP Grey just needs an army of minions/interns/slaves to help him create content.

Why Youtube on TV is not Mainstream

For about 18 months now, I have been watching Youtube on my TV.  At first, I was using youtube-dl to download the shows I liked, and I used a computer that was hooked up to the TV to watch those episodes.  About 10 months ago, I purchased 2 MK802 II's and retired the computer that was hooked up to the TV.  This new interface was usable, but just barely.  As time went on, the Youtube Android app got more and more usable on the TV, but it is still pretty rough for non-technical people.

Other TV devices that support Youtube tend to have similar issues.  The interfaces just aren't designed around navigation from your couch.  Unless you subscribe to channels that are more episodic, it is almost impossible to find content that you want to watch.  The interfaces are designed to allow you to watch that one funny video that you want to show a friend when they come over.

Youtube (or other similar services) won't catch onto the mainstream unless the interface gets designed in a way that competes directly with DVR and On Demand interfaces.  What is funny, is this shouldn't be too hard, since most of those existing interfaces suck!  Google doesn't seem to be taking the interface problem seriously.  As an example of that Google recently released Chromecast.  Chromecast is a cheap HDMI stick that allows you to stream video from your phone/tablet/computer.  Would your grandparents want to go into another room just to pick out what show they want to watch?  No.  You need the interface to start from the TV.

Youtube also has the problem of storing three different types of content: the one off viral videos, the vlogs and the episodic shows.  When people think Youtube, they usually think of the first two.  For someone who just found out about shows like Tabletop or Crash Course, it can be very hard to find similar content.  The TV interface for Youtube should allow you to enter in your interests, then try to match shows/channels that you might be interested in.  From there, you can try out an episode and subscribe after if you like it.

An interface that not only feels more natural for TV watchers, but allows you to more easily find content that you like is the only way Internet-based content will ever tackle the Big Media TV model.

Subbable and the Future of Internet TV

Have you subscribed on Subbable yet?  I have.  I have $1 banked for Crash Course and $1 for CGP Grey.  For those of you don't know, Subbable is a site that allows you pay for "subscriptions" to free Youtube channels.  You may be wondering what the point is?  Why would I pay for something that is free?  The description I gave is a bit over simplified.  Although I love over simplified descriptions, I'm going to go deeper into this one.

Subbable is an attempt by "the brothers Green" to try to make some of the awesome Youtube channels financially self-sustaining.  The idea is if the content is of high enough quality, people will want to pay for it.  As long as enough people pay a small amount of money, these projects might bank enough money to become self-sustaining.  Subbable tries to be a bit like Kickstarter by allowing content creators to set goals and give rewards at certain milestones.  The goal is to allow "alternative" media to find an "alternative" funding source to prevent the degradation of quality in favor for bland mass media.

The whole idea is advertising-based income values the number of views, not the quality of the video.  In a perfect world, the number of views should correlate with the quality of the video, but sadly, it does not.  This can be proved by a number of crazy facts:

1) The Kardashians have multiple hit shows
2) John and Kate Plus Eight was on "The Learning Channel"
3)

This Kickstarter-like setup allows people to directly support the content creators that they feel create the content you most want to see more content from.  As more and more creators get onto Subbable, you can take a budget of lets say, $5/month and split it among those creators.  This style allows those frustrated with the current content offering to put their money where their mouth is.  Eventually, there are more people who like the poor quality content, so Subbable could degrade like regular TV did.

Another problem with Subbable is that it doesn't tackle the distribution problem that really needs to happen to take on Big Media.  Don't get me wrong, Subbable is a great (and needed) step towards that, but most people who watch Crash Course and CGP Grey watch those shows on their computers.  I'm one of the few that actually watches Youtube shows on my TV.  Watching Youtube on a TV is definitely bleeding edge right now.  It is not ready for prime time.  To me, it comes down to two things that still need to happen: better distribution and more (longer) content.  I will talk about those items in future blog posts.