When you're doing it wrong...

How do you know when you're doing it right? Most of the time I know I'm doing it right is when it feels like I'm always hitting my goals, and it's getting easier than it was yesterday. Although that might be a little lie I tell myself because it might just be because I know when I'm doing it wrong, and how bad that feels. If I don't have those bad feelings I know I must be doing something right.

Here's a great example of doing it wrong. I'm at a place that loves to branch code. Most of the time they are branching because the business demands a release, but they have such a large team in order to keep everyone "busy-ish" they have to branch. They have an idea they're doing it wrong, but they don't really have a clue as to how to do it right so they just do what they know. The developer's don't like branching, but the business doesn't give them much choice.

Problem is refactoring is important because the code base is pretty hard to work with. Now when they add multiple branches + refactoring + big team = double black diamond level of difficultly in the merges. So that's bad, but another side effect is when a merge is going on it prevents people from modifying the repository. Nobody can use the source control system while this is happening. It's an all stop. One of these merges is going on it's 5th day. That 5 days where no one has integrated their changes, built all of the code, or synchronized with other people's changes. Now all of a sudden the choices to use SCM, continuous integration, refactoring, and small agile practices is really loosing it's benefit. One developer suggested we send around patches to each other while the merge is going on. We specifically picked a SCM system so we don't do that. Once the merge is done the SCM system is going to hit with tons of changes, and when something breaks functionality they won't be able to easily resolve it because of the volume of changes. Now quality is suffering directly because of relentless branching.

Funny thing is I can't think of anyone out there that suggests branching as a technique for achieving quality. However, there are countless examples from experts that generally agree using SCM, continuous integration, refactoring, and small changes help overall quality. Why are we doing something that sacrifices those best of breed practices? Now the guy with the "big picture" view seems to believe it's the actual code quality is to blame for productivity problems, and quality issues. He thinks more code reviews, and education is in order about how to write "good" code will right the ship. At some level business is just throwing crap over the wall without any real conversations.

This is what I call an "everything is arduous and ridiculous" environment. Everything about this place feels over the top hard. Why don't people around me seems to realize this is the ridiculous way to operate? Haven't they ever had that effortless feeling of productivity? How you're always the man, and it's just right? Sure this works in that we are producing a product very slowly, but it doesn't feel like a success. Is it luck? Is it innate to the problem you're trying to solve? Well...maybe.

Sure some problems are harder than others. Building Google mail is harder than building an android app. But, I've been on some pretty nasty android apps. Which tells me there is a way to make an easy problem hard, and hard problem easy. So what are we doing that makes this problem so hard?

Sometimes it's not being smart enough. We all love algorithms, and finding that simple algorithm that just makes the problem go away is sublime. That's what we all fell in love with if we have any formal training. But, those types of problems are far and few between. Mostly what we do is slog crap from one database, slap it on the glass, then slog the new crap back into the database. Rinse and repeat 1000x and you've got a product. There is no algorithm that makes that easier. If there's no algorithm then what is it?

Technique. There's a difference between algorithm and technique, and what types of problems they are best suited for. Technique isn't going to come up with map reduce. That's algorithm. Technique is your choices for what you're going to use, and how you're going to use it so the problem is easier. Technique is also about how you choose to define the problem which means technique comes before algorithm. How can you choose an algorithm if you don't know what your problem is?

Technique breaks down into two parts. Choosing a set of tools and processes, and how you apply those tools or processes. Technique extends past the end product into the support systems that nurture how that end product is created with bug tracking systems, source control management systems, continuous integration, user forums, etc. And, those choices can have a greater effect on the end product than what you put into the product. Just reread the example above for justification.

To some degree, we place too much emphasis on tool choice because how you apply it can undermine the choice of using that tool. If your technique doesn't match the tool the tool will never matter. Have two bug tracking systems because one group doesn't want to give up their existing one. Been there, real story, doesn't work, definitely doing it wrong. (Actually same place as the example probably could have a book of "doing it wrong" ideas from this place). As in the example at some point application of those tools made the choice of SCM moot.

In the end we need to discuss technique more passionately than specific technologies. The two do go hand in hand, but it's the technique in the end that makes the difference. So how do you know you're doing it right? When technique matters more than technology.

Flexjson meet Android

Flexjson 2.1 now supports running Flexjson on Android. So I thought I'd show a quick example of using Flexjson in an Android application. Hopefully this will spark some ideas about what you can use Flexjson for in your own application. I'm going to start simple creating a quick Android app that pulls recipes from Puppy Recipe, parses it using Flexjson, and displays it in a list. Let's get started.

Recipe Puppy has a very simple REST API, almost too simple, that returns responses in JSON. Recipe puppy allows you to search recipes by the ingredients contained within by using a URL parameter i. Individual ingredients are separated by a comma, and URL encoded. Here is a simple example:

http://www.recipepuppy.com/api/?&i=banana,chicken&p=1

Exciting isn't it? If you click that link you'll see the JSON response. It's a little hard to read like that so here is a simple break down with a little formatting:

{
   "title":"Recipe Puppy",
   "version":0.1,
   "href":"http:\/\/www.recipepuppy.com\/",
   "results":[
      {
         "title":"Chicken Barbados \r\n\r\n",
         "href":"http:\/\/www.kraftfoods.com\/kf\/recipes\/chicken-barbados-53082.aspx",
         "ingredients":"chicken, orange zest, chicken, banana, orange juice, brown sugar, flaked coconut",
         "thumbnail":"http:\/\/img.recipepuppy.com\/602538.jpg"
      },
      ...
   ]
}

This is pretty straight forward. We have a little header and what we really are interested in results property which is an array of recipe objects. So we'll create two simple Java classes to map those data members. RecipeResponse for the header portion, and Recipe which is the object contained within "results" property.

Here are those objects:

public class RecipeResponse {
    public String title;
    public Double version;
    public String href;
    public List<Recipe> results;

    public RecipeResponse() {
    }
}

public class Recipe {

    private String title;
    private String href;
    private String ingredients;
    private String thumbnail;
    private Drawable thumbnailDrawable;

    public Recipe() {
    }

    public String getTitle() {
        return title;
    }

    public void setTitle(String title) {
        this.title = title.trim();
    }

}

In the Recipe object I actually created a Java Bean with getter/setter, but I didn't include most of those methods. I did make a point to show the setter for the title property. Turns out some of the data coming out of recipe puppy contains extra newlines characters in the title. To get rid of those I'm doing a trim() in the setter. Flexjson is smart enough to call the setter method if you have defined it instead of setting values directly into the instance variables. However, if you use public instance variables it will set values directly into those too. This was a fix made in 2.1 with respect to using public instance variables during deserialization process. You'll be happy to know it works now.

So let's jump to the usage of Flexjson in the android code. So we create a RecipeActivity that contains a List to display the recipes. We're going to look at the AsyncTask that loads the data using Flexjson. Here is the full code for that:

        new AsyncTask=<String, Integer, List<Recipe>>() {

            private final ProgressDialog dialog = new ProgressDialog(RecipeActivity.this);

            @Override
            protected void onPreExecute() {
                dialog.setMessage("Loading Recipes...");
                dialog.show();
            }

            @Override
            protected List<Recipe> doInBackground(String... strings) {
                try {
                    return getRecipe( null, 1, "banana", "chicken" );
                } catch( IOException ex ) {
                    Log.e( RECIPES, ex.getMessage(), ex );
                    return Collections.emptyList();
                }
            }

            @Override
            protected void onPostExecute(List<Recipe> results) {
                if( dialog.isShowing() ) {
                    dialog.dismiss();
                }
                Log.d( RECIPES, "Loading " + results.size() + " Recipes" );
                recipes.setList( results );
                new ThumbnailLoader( recipes ).execute( recipes.toArray( new Recipe[ recipes.size() ]) );
                Log.d( RECIPES, "Loaded " + recipes.size() + " Recipes" );
            }

            protected List<Recipe> getRecipe( String query, int page, String... ingredients ) throws IOException {
                String json = HttpClient.getUrlContent( String.format( "http://www.recipepuppy.com/api/?q=%s&i=%s&p=%d",
                        query != null ? URLEncoder.encode(query) : "",
                        ingredients.length > 0 ? URLEncoder.encode(join(ingredients,",")) : "",
                        page ) );
                RecipeResponse response = new JSONDeserializer<RecipeResponse>().deserialize(json, RecipeResponse.class );
                return response.results;
            }
        }.execute();

The method your probably most interested in is getRecipe(). This method formats the URL we're going to load. It then loads that URL and passes the results returned as a JSON block to the JSONDeserializer. JSONDeserializer will take a JSON formatted String and bind that into a Java object. In this example, we're binding into a RecipeResponse object. Here is how that is done:

RecipeResponse response = new JSONDeserializer<RecipeResponse>().deserialize(json, RecipeResponse.class );

A single line of code does that. The deserialize() method performs the deserialization and binding. The first argument is the JSON String, and the second is the top level class we want to bind into. Notice we didn't have to mention anything about Recipe. Flexjson is smart enough to use the data types from the top level object to figure out any other data types contained within. So if you refer to the RecipeResponse.results instance variable you can see the List data type with a generic type. Flexjson will use generics whenever possible to figure out concrete types to instantiate. Of course polymorphism, interfaces, abstract classes, and the like causes issues with this, but we're not going into that right now. See the Flexjson home page to find out more.

You'll notice the RecipeResponse object is returned fully populated with the JSON data, but we're really only interested in response.results so we just return that. It'd be nice if Recipe Puppy returns how many total pages there were in the header (hint, hint) so that it was more interesting. Anyway it is beta. That array is then added to the ListAdapter and displayed on the screen.

Other things Flexjson could be used for is saving state by serializing objects to JSON, and then deserializing when Activities are reconstituted. This can be easier than writing ContentProviders to dump stuff into the database. One of my biggest gripes with Android is how between pages objects can be reliably sent because Intent's require you break everything down to primitives. With Flexjson we can just simply serialize an object put that in the Intent, and then deserialize it on the other side. So no more boilerplate code to flatten your objects.

Here's a simple example serializing our recipes to the disk:

        File f = app.getFilesDir();
        Writer writer = new BufferedWriter( new FileWriter( new File( f, "recipes.json") ) );
        try {
             new JSONSerializer().deepSerialize(favorites, writer);
             writer.flush();
         } finally {
             writer.close();
         }

Now I know there are people worried about performance, but timing the following code this ran on device in less than 40ms which is within the acceptable bounds for UI performance. If you need more performance you can cache the JSONSerializer/JSONDeserializer instance which optimizes data type mappings so it doesn't recompute those when serializes and deserializes. As always measure, measure, measure.

You've gotten an introduction about how Flexjson can make it easier to work with JSON data with Android.

Caringorm is Architectural Poison

No one has ever accused me of shying away from sensational titles, and now's not the time to get timid. I must confess I've never been a fan of Caringorm. My first impression of Caringorm was it's over engineered. Why are there so many layers? Isn't that going to just slow you down having to develop a View, Command, Delegate, Service, etc for every round trip I make to the backend? Now that I'm working on a project that has gone horribly wrong I see how Caringorm architecture directly contributed to the problems. We've been called in to straighten out the mess and put down a more suitable architecture. After understanding what the team has done I begin to see the techniques, that Caringorm purports as best practices, create more work for yourself the longer you use them.

Software architecture should organize your work so you can work at a higher level related to your problem you are trying to solve. It does this by fostering reuse in your code. It should allow you to reuse what you did yesterday to apply to today's problem reducing the work required get work done. As your project grows the only way you can move quickly is through reuse. Without reuse the work grows exponentially to the point where value can't be delivered. How long this takes before your code base becomes unproductive? Ten releases? 10 Years? I've seen it happen in 1 release, and less than 1 year.

Key signs this has happened in your project is talks about rewriting your application or major refactoring. Other signs come in when your customer says that should be easy, and then developing it takes significantly more time than you'd expect. Bad architecture robs your team's performance to deliver value. If this goes on too long your project will get scrapped and if you're lucky you'll be allowed to start another project. Most likely you won't because the business will be putting your project in maintenance mode while they spin up the "solution". Good architecture is quite the opposite. Easy things are easy and hard things are possible. At the core of this is the level of reuse in your project.

Caringorm doesn't foster reuse. It stalks it, attacks, leaves it dead, and poisons the earth to keep it from ever fostering. At the heart of this is the age old singleton problem. Singleton's are seriously bad technique, and I wish every developer out there understood this. Using singletons to limit an instance to a single instance is not all together bad, but using it as a locator pattern is where the serious issues began to make your code single use. Unfortunately you can't limit a singleton to the good parts without accepting the very serious downsides, and this is the reasons I try and void them at all costs because the downsides are that damaging. Cargingorm has no problem using the ModelLocator (which is a singleton) in your views (mxml). And there is no amount of other techniques you can introduce to overcome the problems that come with this. I don't care if you're using Code Behind, Presentation Model, or whatever. If you use a singleton in your views you can't reuse them. Anything that directly references singletons becomes single purpose in its use as well, including anything referencing those objects, and so on and so on.

Why is that a problem? Well consider if we wrote DataGrid with the same techniques Caringorm purports as acceptable practices. Let's say DataGrid.dataProvider was hard coded to look in ModelLocator.getInstance().dataProvider. Now how can you have two instances of DataGrid in your program pointing at two different dataProviders? You can't. And this is precisely the problem that leads to serious architecture problems with Caringorm programs. Now throw in calls to getController().eventManager.addEventListener() in your views and you have a serious recipe for disaster.

You might find my example contrived so let me describe a more real world scenario. Say you have a signup process that people fill out on your site and you have a view that represents the information you want to gather. In that view you're using the ModelLocator. Now the customer wants to add a new way to sign up because their doing an email campaign, and they'd like to pre-populate that form from details like the email address and ad campaign number to track it into the view. Unfortunately, ModelLocator makes it difficult to put two different models into your view because it's hard coded to one. What would have been an easy task by instantiating another instance of your view has turned into creating another view from scratch. So let's say you need to do this fast and you copy the view and makes the changes to create two different views. Then the customer wants to add a field to both views. Now you need to update two places in your application. This is precisely what I mean when I say Caringorm creates more work for you. Over time if enough of these exist in your application your productivity will drift to zero because maintaining all of it too much work.

Now hopefully I convinced you that the patterns Caringorm suggests are not helping you. And, you decide to banish ModelLocator from the views. However, the problem of geting something from the model and bound into the view still exists. So what part of the Caringorm architecture will interact with the ModelLocator and the view? Normally this would be the Controller in a traditional MVC pattern. In Caringorm the Command is suppose to be this part, but it doesn't have access to the view. Therefore, how will it set the data properties on the view? You could do some gymnastics by passing references through the FrontController into the Commands, but at this point I'm taking some serious liberties to modify Caringorm's architecture design to make it work. If the architects of Caringorm had realized this then their examples would have shown how to do this.

I'd like to think that as an industry we're working towards a shared understanding about the dangers of singletons, and if that were true I'd expect to see a drop in the number of projects using them. I'd expect to see a reduction in the number of frameworks employing singletons as an instance locator pattern. However, it's quite the opposite. Most developers don't see a singleton and get that tingling sensation that something bad is about to happen. But, singletons have all the same problems that global variables do and by in large most developers realize global variables are poison if not very carefully used.

Caringorm is like EJB of Flex. Over engineered. Expect there to be serious changes to Caringorm in the future to save the marketing that Adobe has done with clients. Just like Sun did with EJB3 for EJB. Sun had over emphasized the benefits (if there were really ever any) to using EJB, and once the community realized EJB was over engineered and more trouble than it was worth. Sun had no choice but to hire Hibernate's creator to design EJB 3.0 and started begging for forgiveness. Adobe will have to do the same.

GreenThread: Problems with Recursive Functions

In previous blog posts on GreenThreads I mentioned that the downsides of using GreenThreads meant you couldn't write recursive functions. In one of the comments I was asked to expand on this idea, and after the comment got so long I figured a blog article might be a better forum for this topic. I'm going to discuss the issues with regular recursive functions, then we'll explore the differences between two types of recursive functions, and potential changes that could be made to help make it easier to write recursive GreenThreads.

Let's start by examining the following function:

public function factorial( i : int ) : int {
   if( i == 0 ) return 1;
   return i * factorial( i - 1 );
}

It's like the "hello world" of recursive functions. What makes a function recursive is the fact factorial() function calls itself in evaluating the value of the function. If we were to run it in a GreenThread there's no way for the system to interrupt the function calls should this function take longer than the length of a frame. For example, say you ran factorial(5) in the GreenThread. The call stack will look like the following:

factorial(5) -> factorial(4) -> factorial(3) -> factorial(2) -> factorial(1) -> factorial(0)

There's no way to let Flash insert a paint in between factorial(3) calling factorial(2) because factorial(3) calls factorial(2) directly.

The way GreenThread framework works is that it handles repeatedly calling your GreenThread until the time has elapsed for a single frame. At that point it let's Flash have control again and then resumes on the next frame repeating this process until your GreenThread says it's finished. This is actually implemented using a big loop outside your GreenThread.

It gets even harder for recursive functions. Look back at factorial() function. Notice that factorial(5) has to compute factorial(4) before multiplying by 5 so it can return it's value. Therefore, it's not possible break out of the function call, allow Flash to paint, then resume within a function so it can multiply by 5 to finish the computation. (Not unless Flash supported continuations, but that's a whole another topic). So now recursive functions can't be interrupted because the function directly calls itself, and depending on how you write your recursive function it's not possible to put a break because of operations that might come after the recursive call finishes.

There are other issues with recursive functions, but they are not possible to use in a GreenThread because of the dependencies between stack frames. However, there is another type of recursion that can help eliminate dependencies. Let's write our recursive function to remove the dependency on local operations:

public function factorial( i : int, accumlator : int = 1 ) : int {
   if( i == 0 ) return accumlator;
   return factorial( i - 1, i * accumlator );
}

Now notice that factorial(5) doesn't have extra work to run after factorial(4,5) runs like we did before. This means factorial(5) could be replaced by the return from factorial( 4, 5 ). In fact factorial(5) == factorial(4,5)! This technique is called tail recursion, and in certain languages it helps make recursion work without growing the stack frames so large iterations don't overflow the stack. Now Actionscript doesn't benefit from this, but this will allow us to work around the second problem we have. We still have our original problem so we'll have to tackle that before we're done.

Now we still have factorial(5) directly calling factorial( 4, 5 ) so Flash can't interrupt the function calls so it can paint. However, what if we had a special call that would delay calling factorial(4,5) so we could do whatever Flash wanted, then it would resume our recursive function.

Well there exists such a function: callLater(). callLater() can be used to schedule a function to be called in the next frame, and in fact from all of the testing done it's safe to use callLater() as a technique for implementing GreenThreads. However, directly using it will suffer from poor performance because of long waits between function calls. So, let's assume there is a new function in GreenThread that acts like callLater(), but achieves better performance. Now our recursive function could look like:

public function factorial( i : int, accumlator : int = 1 ) : Boolean {
   if( i == 0 ) return false;
   return invokeOnThread( factorial, i - 1, i * accumlator );
}

Now invokeOnThread() doesn't exist in the current code base, but it could be written. Actually the accumlator would probably be best served as an instance variable within your GreenThread, and we'd need to change some more features to fit within the framework. Assuming that it is we could support recursive functions given these constraints:

• You must write your recursive calls so they conform to tail recursion.


• You must use invokeOnThread() to recursively call your function.


• You must conform to the contracts of the GreenThread framework.

The upside is the ability to think recursively. While every algorithm can be expressed either in iteration or recursion it's not easy to convert between each form. Some algorithms are easier to express using recursion and can be very hard to write iteratively. The downside is the requirement to write tail recursive algorithms which can be difficult for the uninitiated, but it's a skill that can be honed.