Java 8: What Streams can do for you.

In my previous post, I wrote about how Java 8 is starting to shift to Declarative programming. Today, I’ll expand on that and talk briefly about how the new Stream API will change the way developers work with data structures ie. Collections. The Stream API is very complex, but at the most basic level, it introduces various operations that perform tasks on a Collection. A Stream consumes a collection and pipelines the data from the collection. The Stream operations can be classified into two types:

  1. Intermediate – these operations take the original stream and produce a new Stream. Intermediate operations need a terminal operation to actually perform work. These operations are Lazy and don’t perform work until the terminal operation is called. Some examples include:
    • Filter
    • Map
    • Sorted
  2. Terminal – these operations consume a Stream, execute the terminal logic, and produce a result.
    • Collect
    • ForEach
    • Reduce
    • Sum, Count, Average, etc

Streams and their operations let us perform data manipulation that used to require Loops, Iterators, and complex inline logic. For example, lets consider the array of numbers in my previous post. This time I want to get the Sum of All Odd numbers.

List<Integer> numbers = Arrays.asList(1,2,3,4,5,6);

Using pre Java 8 approach I’d loop and add up all items that are Odd. If I were in post Java 5, I could use an Iterator.


int sum=0; // The sum of all odd numbers
for(int x=0; x<numbers.length(); numbers++){
 if(numbers[x] % 2 !=0)
  sum += numbers[x];
}

Using Streams, I can write code that is easily readable and more concise:


int sum = numbers.stream()
                 .filter(e -> e%2 != 0) // Intermediate op
                 .sum();  // Terminal op

This code is easier to maintain and it expresses the intent of the code without any comments. This is a simple example, but we need a more complex example to show the real power of Streams.

Most of the time you loop a Collection you actually want to manipulate the objects in the collection and do something more complex. Consider the same array full of numbers. This time, we want to make an HTTP post request for each element in the array, and Post the int to a REST endpoint.

 


numbers.stream()
       .forEach(e -> doHttpPost(e));

public void doHttpPost(int e){
  ... // DO HTTP POST HERE. Logic not relevant for this discussion.
}

We have no intermediate operation to perform, so we just use the forEach() terminal operation and call the doHttpPost() method for every element in the Stream. Now, lets say our requirements change and we only want to send the Odd elements that are greater than 1.  Just add both .filter() intermediate operations (you can also combine them):


numbers.stream()
       .filter(e -> e%2 != 0) // Satisfy our new "Odd only" requirement
       .filter(e -> e > 1) // Satisfy our > 1 requirement
       .forEach(e -> doHttpPost(e));
}

Now, lets pretend our REST service provider now requires us to wrap each int in a Number object. This is when the map() operation is powerful. Map is an intermediate operation that lets you create a new Stream. So, we filter only on Odds greater than 1, then create a new Stream of Numbers, and finally perform the HTTP Post.


numbers.stream()
       .filter(e -> e%2 != 0) //Odds only
       .filter(e -> e > 1) // Greater than 1
       .map(e -> new Number(e)) // New stream
       .forEach(e -> doHttpPost(e)); //do Post

// Example Number Object
public class Number(){
  int num;

  public Number(int newNum){
   num=newNum
  }
}

Finally, lets pretend that sequentially processing HTTP posts is the cause of a system bottleneck. We need a way to improve performance. Streams can actually pipeline the data in parallel, all you have to do is change the single stream() method to parallelStream(). The entire operation is now performed in parallel:


numbers.parallelStream()
       .filter(e -> e%2 != 0)
       .map(e -> new Number(e))
       .forEach(e -> doHttpPost(e));

As you can see, Intermediate stream operations can be chained to perform all sorts of logic. Converting from Sequential to Parallel processing is simple and does not result in a complete rewrite of the code logic. One thing to note is that you must be careful of the parallelStream() method, as it creates Threads and shares the common thread pool.

Obviously these are simple examples but I hope they demonstrate the power of Java 8 Streams and Declarative programming.

Java’s Functional and Declarative Evolution

Java 8 is the largest change in the Java platform to date. One of the primary goals was to move the language from an imperative model to a declarative model. Before Java 8, software engineers wrote code that explicitly modified the state of the program and provided instruction on how all operations should be performed. Let’s discuss the evolution of Java development. Consider this example:

We have an Array collection with a bunch of numbers {1, 2, 2, 3, 5, 8} and need to find the “sum of all numbers.” Using Java 4 or lower (imperative) you have to loop through the array like this:

int sum=0;
for(int x=0; x<array.size(); x++){
// do the summation
 sum+=array[x];
}

This code explicitly loops through the array, one element at a time. Java 5 simplified things a bit, with the introduction of the more concise For loop:

int sum=0;
for(int x:array){
 sum+=x;
}

So no longer care about the length, we just loop through the collection. But if you compare this to the previous code, it’s still the same approach – we tell Java to explicitly loop through each and every element in the array. We still need the same amount of code to do it (minus some < and ++ operators) and readability is more or less the same. We wrote more logic just looping the array than our actual business logic.

We can also use the Iterator pattern like this:

int sum=0;
Iterator i = array.iterator();
while(i.hasNext()){
sum+=i.next();
}

If we compare all three options, they’re all similar amount of code and complexity. The iterator pattern actually introduces even more objects for the JVM to manage.

Onto Functional Style
With the introduction of Java 8, we get two things: Lambadas and the ‘forEach’ method on all Collections. ForEach provides us with a reference to all items in the collection. The implementation is specific to the type of collection. We just have to call the ForEach method. Java will execute and provide us the implementation. This is a classic example of polymorphism. Here’s how we use it:

int sum=0;
array.forEach(n -> {
//implementation here
sum+=n;
});

With this approach, we don’t care how the array is looped — there is no need to manage iteration values or indexes. ForEach will handle it for us and call the body of our function, for every element in the array (commented area). The code that is in the forEach: (n-> { … }) is lambda notation.

This lets us focus on implementing the solution to our problem (summing the contents of the array), rather than worrying about how to access our Array. It’s the job of the Java API to handle things it *already* knows how to do.

Compared to our Java 5 implementation, the functional style is easier to read, is less complex, and helps us focus on code that represents our business logic.

More Features
This is just an example of how Java 8 will change the way developers write code. I’ll talk about further improvements including the Stream / Map / Reduce / Predicate APIs in my next article.

Google Glass: Development & GDK!

Learning the Glass Development Kit was one of my immediate goals after getting Glass. So, just like learning any new Platform, I built a  “HelloWorld” LiveCard application 🙂

The GDK itself is not finalized and is still in preview stages. Google has some decent samples here.  Turns out, getting your development environment ready is a little tricky. The Quick Start Guide is a great place to start, but I’ll summarize the entire process here:

  1. Download the ADT Bundle. This contains the SDK Manager and Eclipse based Android Development Tools (ADT) IDE.
  2. Unzip the bundle. Run the SDK Manager and install the supported Glass Android version (currently, 4.4.2). Make sure you include the SDK Platform and Glass Development Preview Kit.sdk-manager
  3. You will need to install the newest Android SDK Build-tool (currently 19.0.1). I accidentally installed 19.0 and ended up with runtime issues.
  4. Finally, install the Google USB Driver. This allows the ADT to communicate with Glass. When installed, it will be located in <your install folder>/sdk/extras/google/usb_driver/.usb driver
  5. Next, you need to prepare Glass itself to accept custom built software. Go into the “Settings” live card, then scroll until you see “Device Info”. Tap once, choose “Debug” and enable it.
    settings
  6. Unfortunately, the provided USB Driver is not compatible with Glass. You’ll have to modify the “android_winusb.inf” file to include Glass entries before Windows will accept the driver. Add the following to the [Google. NTx86] and [Google. NTamd64] sections:
    ;Google Glass
    %SingleAdbInterface% = USB_Install, USB\VID_18D1&amp;amp;PID_4E11&amp;amp;REV_0216&amp;amp;MI_01
    %CompositeAdbInterface% = USB_Install, USB\VID_18D1&amp;amp;PID_4E11&amp;amp;MI_01
    

    Note: The actual VID , PID, REV, and MI values must match what your actual hardware device ID says. You can get your specific values by plugging Glass into your USB port. Open up your Device Manager, browse to the Glass device that has a missing driver (marked with !). Right click, goto Properties –> Details Tab –> Hardware IDs dropdown.
    adb device

  7. Install the USB Driver to Windows.  Open up your Device Manager, browse to the Glass device. Right click and choose Update Driver. Browse to the above folder (Step 4) and proceed with the install.
  8. With Glass plugged in, go back to <your install directory>/eclipse and start Eclipse. Switch to the DDMS perspective (Window –> Open Perspective –> DDMS).
  9. You should see Glass in the Devices tab.
  10. Now you’re all set!

Comparsion: Struts 2 vs Spring 3 MVC

Apache Struts 2 and Spring MVC are two of the most popular Java web frameworks today. I’ve used both APIs extensively and wanted to share a quick comparison.

In Struts, the object that handles a request and routes it for processing is called an Action. In Spring MVC, that object is referred to as a Controller. Actions and Controllers are pretty much the same thing – they take input, process it, and return some sort of response.

The one major design difference is that by default, Struts 2 Actions are newly instantiated every time a request is made, whereas in Spring MVC the default behavior is to act as a Singleton. Spring MVC Controllers are created once and held in memory/shared across all requests. Note, you can change this behavior (scope) to request or session but we’ll talk about that later. This is a major difference to keep in mind when designing applications that need to be thread-safe, database oriented, or other share-able transactions.

Struts 2 Actions:
The basic Struts 2 Action looks very similar to any normal Pojo but with an execute method:


public class TestAction{
   public String execute() {
       return "success";
   }
}

This method just returns the name of the result as defined in the struts.xml configuration file.
The struts framework will check the XML for the configured return path and handle forwarding to the proper view (usually a JSP page).

Spring MVC Controllers:
The basic Spring MVC controller only requires that you tag it with the @Controller annotation. Then you should use the @RequestMapping annotation to specify what URL patterns the controller will respond to. You can use this annotation at the class level or method level or a combination to achieve your needs.


@Controller	// Controllers are singletons by default
@RequestMapping(value = "/actions")
//@Scope("request")	// This will make spring instantiate a new object for every request
public class ControllerTest {
  
  @RequestMapping(value="/index", method=RequestMethod.GET)
  public ModelAndView index(ModelMap model) {
    ModelAndView mv = new ModelAndView("index", "model", model);
  }
}

The above example shows the index method that would respond to a GET request in this URL Format: /actions/index. We could easily change the RequestMethod to POST and it would ONLY respond to POST requests.

Struts 2 Configs:
As you can see, the MVC concept is pretty similar between both frameworks. Now, lets compare the configurations required.

Struts 2 only requires a single ‘struts.xml’ config file to configure the framework. A typical action/result routing configuration looks like this:

<action name="TestAction" class="com.TestAction">  
            <result name="success">/pages/success.jsp</result>
            <result name="input">/pages/input.jsp</result>
            <result name="fail">/pages/failure.jsp</result>
</action>

In the example above, we returned the “success” result. So this Struts 2 would forward us to success.jsp as the view.

Spring MVC Config:
Spring MVC requires a ‘servlet-context.xml’ file. As seen above, the actual result routing/configuration is entirely completed within the Controller classes via annotations. Here’s snippetts from the file:


<!-- Configures the @Controller and annotation programming model -->
	<mvc:annotation-driven />

<!-- Forwards requests to the "/" resource to the "welcome" view. Use this to setup a default action -->
	<mvc:view-controller path="/" view-name="welcome"/>

<!-- Resolves view names to protected .jsp resources within the /WEB-INF/views directory -->
	<bean class="org.springframework.web.servlet.view.InternalResourceViewResolver">
		<property name="prefix" value="/WEB-INF/views/"/>
		<property name="suffix" value=".jsp"/>
	</bean>

The last bean entry is the most important. In our above example we returned the “index” model and view. Spring will automatically prefix and suffix the view resulting in “/WEB-INF/views/index.jsp” as the JSP that handles the request. Of course, you can configure the path to whatever you desire.

You can find a fully working servlet-context file anywhere on the internet or you can contact me and I am happy to provide you an example.

Note, most development teams will then choose to deploy the full Spring architecture and have the Core Application Context enabled as well. For example, this would be necessary if you wanted to have Spring manage your Data Layer via JPA / EntityManager. Or if you wanted to have Spring manage and inject your Beans across your application.

Summary:

Either framework is a good choice. I suggest reviewing the exact needs of your application before choosing a Framework. I’m available to help if there is any need!

Thanks for reading.

How do you get a Oracle Sequence value with JPA?

I have to say, JPA 2.0 is pretty awesome – it does a great job of Entity Relation mapping. Overall I really like the ORM framework and I’m glad its built into Java now. However, its by far not a perfect solution to all database related tasks. For example, some of the generated SQL isn’t the best. But that can easily be solved by writing your own SQL using NamedQueries.

Another common task is to generate records with primary keys that are sequenced based. This is easily accomplished by using the @GeneratedValue tag and a @SequenceGenerator

@Id
@GeneratedValue(strategy = GenerationType.SEQUENCE, generator = "SeqGen")
@SequenceGenerator(name = "SeqGen", sequenceName = "SAMPLE_SEQUENCE", allocationSize = 1)

But what if you wanted to generate a value that is based off a Sequence? You don’t want to tie this value to a Primary Key, instead maybe you want to populate another field with it? Or generate a confirmation id, etc.

As far as I know, there is no annotation that will query a Sequence and simply return its value. I ended up using a query to return the next Sequence value:

Query q = em.createNativeQuery("SELECT SAMPLE_SEQUENCE.nextval from DUAL");
BigDecimal result=(BigDecimal)q.getSingleResult();   
return result.longValue();

Anyone know of a better way to accomplish this?

Netbeans Rant

Netbeans is an awesome IDE (my fav one), but one thing that annoys me is the lack of support for JAR files that depend on Native Libraries. Java Native Access allows Java APIs access to functionality bundled in a DLL. With JNA you can call directly into a DLL and access its methods. Most often used for integrating with C/C++ code.

One way to set this up is to include a MANIFEST.MF file in your JAR. You add a line like this:

Bundle-NativeCode: [Directory to the DLLs] 

Now you can distribute the JAR and required DLLs easily. With Eclipse this is easy – you simply add the JAR to your Project Build Library. Then expand the node and configure the “Native Library location” option to point to your DLL(s).

However, in Netbeans if you try to add a JAR that has DLLs specified, the IDE immediately crashes and throws a “Class not found” error. As far as I can tell – there is no way at all to setup a Native Library JAR file in Netbeans. If anyone has done this please let me know how! 🙂

Your options are:
1. Use Eclipse
2. Add the DLL path to the Java executable like so:

java -Djava.library.path=[path to native lib folder]

Usually adding the DLL(s) path to the PATH environment variable does the trick. Then go into the MANIFEST.MF file and remove the line for “Bundle-NativeCode”. Finally, add the JAR in Netbeans and it will work!

Create a MD5 Hash with no 3rd Party Libraries

I generally like to stay with built in Java libraries unless a 3rd party API offers significantly better functionality. An example I recently came across is creating MD5 Hashes in Java. There are many APIs available (Fast, Apache Commons) and they are probably a little faster than Java’s own libraries. However – staying with the Java 6 API was my goal.

I used the java.security.MessageDigest class and some helper methods. Check out the code below to generate a MD5 hash and represent it in Hex/String format.

It’s simple, just pass in the string you want to Hash. SALT is a secret string saved somewhere else:

public String generateMD5Hash(String function){

        MessageDigest digest= null;
        try {
            digest = java.security.MessageDigest.getInstance("MD5");
        } catch (NoSuchAlgorithmException ex) {
            ex.printStackTrace();
        }
        digest.update((SALT+function).getBytes());

        byte[] messageDigest = digest.digest();

        String md5 = toHex(messageDigest).toString();

        System.out.println(md5);

        return md5;       
    }

And the toHex function here:

public static String toHex(byte[] bytes) {
        BigInteger bi = new BigInteger(1, bytes);
        return String.format("%0" + (bytes.length << 1) + "x", bi);
    }

Forrester Says Java is a Dead-End – Um, what??

I just read a post by Mike on Forrester’s Blog titled “Java Is A Dead-End For Enterprise App Development”.  While some of his points are valid, I disagree with his reasons. Here’s why:

  1. Mike says “Java was at the right place at the right time.”
    • I don’t see how this leads to dead ends. Having the right product at the right time is how successful businesses are made.
  2. Then he goes on to point out various deficiencies in the Java platform and why the industry should “move forward”
    • Java is a 20 year old language based on C++” – I fail to see how this is relevant to his argument.  Java has evolved through 6 major revisions, approaching 7 very soon.  Its true the focus has moved all over the place but I think most people will agree that the API is in the best state yet. Think about  Spanish and French, which are hundreds of years old, based on Latin. Does that make them obsolete? Additionally, his facts aren’t correct. Java was released in 1995 making it 15 years old.
  3. “Java frameworks prove complexity” – Mike says that popular frameworks (Hibernate, Spring, etc) cause complexity. 3rd party APIs do add complexity – thats true for any language. But take a look at the purpose of these frameworks. They solve problems that are common to all n-tiered projects by providing a common framework. The community has proven and accepted them and developers can use the same framework across projects. Imagine if these frameworks didn’t exist. Should Oracle buy out these technologies and bundle them into the JDK – Thats another discussion.

Don’t get me wrong- Java is not a perfect language/platform and I do agree with this point:

  1. “Java bungled the presentation layer” – I don’t think its so much as bungled, but Java never really focused on providing pretty presentation. I agree that the desktop APIs have really sucked. But on the web, Java provides a way to send data to the presentation and leaves the actual pretty stuff up to 3rd party tools.

Its pretty clear to me that Mike wants Java to be one big bundled PRODUCT. Why? Because thats what Forrester does – they review products. They compare Java to BPM and other COTS products. That’s not even a valid comparison.

They want the Java platform to do everything and be a one stop shop for development. On the contrary, that’s not what Java is about. The Java concept is: Openness and Flexibility for framework choice.   Not everyone needs or wants to be tied into an entire platform when they only need a small piece of it …

Google Translate Recap

I’ve been implementing a Java API that can interact with Google Translate for integration into a 3rd party application. I wanted to share some thoughts on the process and see if anyone can provide some feedback.

  1. As far as I can tell, Google does not provide a prebuilt API for interacting with the service
  2. Google expects you to query the service via simple URL parameters and parse a JSON response
  3. All you need to do is Post a request to the following URL: https://ajax.googleapis.com/ajax/services/language/translate?v=1.0
  4. Then attach 2 parameters with their values: q (The text to translate) and langpair (The languages to and from, separated by a | character)

An example URL looks like this: https://ajax.googleapis.com/ajax/services/language/translate?v=1.0&q=hello&langpair=en|fr. Building the Java API was actually pretty easy.

  1. We used the Java class: HttpURLConnection to handle the actual URL query and response
  2. Google responds with a JSON encoded string, so we needed an API to parse and create Java Objects out of the string. I chose GSON because I figured it would work best to use Google’s own products.
  3. The JSON returned by Google looks something like this:

         "responseData": {
              "translatedText": "Hola, mi amigo!"
         },
         "responseDetails": null,
         "responseStatus": 200
    }
    
  4. Next, you need to turn the JSON string into GSON / Java objects. This process is called deserialization. A pretty simple example from my API is below:
  5. JSONResponse data = new Gson().fromJson(rawJSONString, JSONResponse .class);
    

    The JSONResponse object is a custom POJO that will hold the data from the JSON string. Gson is the Google implementation of JSON and you simply call the fromJson() function to deserialize the JSON string, rawJSONString, and construct the object.

  6. Once you have the final GSON object, you can treat it like any Java POJO and ask it for properties. Note, we could have simply proceeded without the String -> GSON object step and parsed the String. However, that approach can get complex and can be very tedious.

Overall, I found the process really easy and quick to implement. I’m not sure why Google doesn’t just provide a downloadable API. Maybe its because they want to push the lightweight REST and JSON formats or remain language agnostic? Most likely, I think they meant the service to be used in a Javascript/Ajax environment and embedded into a Web application.

Contact me if you would like to know more or have ideas on a better way to do this 🙂

You can check out their online documentation for more information.

Java as Stored Procedures in the Oracle Database

I attended an Oracle Technology Network event in Reston, VA yesterday. Pretty cool event – they talked about the latest 11g related technology stacks. One of the more interesting techniques they talked about was Java Stored Procedures. If you haven’t seen this technique before, it actually loads and compiles Java source code into the Oracle DB.  Once Java source is loaded, you can reference and execute them via Stored Procedures or PL/SQL. They execute entirely inside Oracle’s JVM.

Here’s how you create a Test class with a single main function:

create or replace java source named “Test” as
public class Test{
public static void main(String args[]){
System.out.println(“Hello”);
}
}
/

You can also use the ‘loadjava’ utility to load entire source files(ex: loadjava Test.java).

Now, you need to wrap and expose the Java class via PL/SQL. Create a procedure named ‘TestProc’ as follows:

create or replace procedure TestProc as language java name ‘Test.main(java.lang.String[])’;

Execute the Java class:

set serveroutput on
call dbms_java.setoutput(50000);
call TestProc();

This will execute the main function in the Test class and print “Hello”. Now the question is:

Why would you ever want Java inside your Database? Doesn’t this break all sorts of design & logic rules?

The truth is, yes it does break many traditional design rules, mainly the one that says keep data and logic seperate.

However, Java is faster than SQL at certain things. These include analytical processing & almost any type of math. PL/SQL is faster at data manipulation and has direct access to the data. If you need to perform many commands that are performance enhanced by Java, then it may actually make sense to put the Java code inside the database!

The one clear advantage is that your Application will save the time needed to context switch when making a database call and waiting for it to return. Instead, you make 1 PL/SQL call which executes some Java and returns with results.

I wouldn’t go overboard with this technique, but there are some scenarios where it does make sense 🙂