NoSQL – One Simple CRUD Example

Here I’d will explain how simple, with the right .jars, is work with NoSQL, in my example will be used the CouchDB, that is pretty simple and easy to install and make it run. Basically just unzip it and run the .bat file.  For any other explanations about NoSQL databases please go to this llink, or to this other one.

.jar’s that you will might need

Couch4j

Json lib

ezmorph

httpClient and httpCore

Here we’ll create one very simple example with simple table and no relationships.

Field name

_id

_rev

EMPLOYEE_ID

FIRST_NAME

FAMILY_NAME

GROUP

FUNCTION

We don’t need to worry with the field types due that the document, is a JSON object so, just strings and you must perform the conversions later on. Here one simple JSON document as example:

{

 "_id":"1",

 "_rev":"1-10",

 "EMPLOYEE_ID":"0001",

 "FIRST_NAME":"MyFirstName",

 "FAMILY_NAME":"MyFamilyName",

 "GROUP ":"JAVA",

 "FUNCTION":"Developer"

 }

CouchDB reserve those fields starting with _ (underscore) to be used to itself, e. g. : _id , and the _id itself is a primary key.

As I’m using the CouchDB as localhost and the default port to access it is 5984, and to do any sort of access into CouchDB you must have one session created, in other words session is mandatory.

   1:  package au.com.nosql.main;

   2:   

   3:  import com.fourspaces.couchdb.Database;

   4:  import com.fourspaces.couchdb.Document;

   5:  import com.fourspaces.couchdb.Session;

   6:  import com.fourspaces.couchdb.ViewResults;

   7:  import java.util.List;

   8:   

   9:  public class NoSQLMain {

  10:   

  11:     public static void main(String[] args) {

  12:        boolean exist = false;

  13:        Database db = null;

  14:        

  15:        // Creating the session

  16:        Session mySession = new Session("localhost", 5984);

  17:        String dbname = "employeeTest";

  18:   

  19:   

  20:   

  21:        // Check if the database already exist, if not create

  22:        // Including one document into my new database

  23:        List<String> dbsNames = mySession.getDatabaseNames();

  24:        for (String dbName : dbsNames) {

  25:           if (dbName.trim().equalsIgnoreCase(dbname)) {

  26:              System.out.println("Already exist the database with name" + dbname);

  27:              exist = true;

  28:              break;

  29:           }

  30:        }

  31:   

  32:        if (!exist) {

  33:           mySession.createDatabase(dbname);

  34:        }

  35:   

  36:        db = mySession.getDatabase(dbname);

  37:   

  38:        // create one document and save it

  39:        Document doc = new Document();

  40:        doc.setId("1");

  41:        doc.put("EMPLOYEE_ID", "0001");

  42:        doc.put("FIRST_NAME", "MyFirstName");

  43:        doc.put("FAMILY_NAME", "MyFamilyName");

  44:        doc.put("GROUP", "JAVA");

  45:        doc.put("FUNCTION", "Developer");

  46:             

  47:        db.saveDocument(doc);

  48:        doc = null;

  49:   

  50:        // Retrive that list of DBs

  51:        List<String> listofdb = mySession.getDatabaseNames();

  52:        System.out.println("Number Total Of Databases: " + listofdb.size());

  53:   

  54:        // Displaying the current number of documents

  55:        int count = db.getDocumentCount();

  56:        System.out.println("Total Documents: " + count);

  57:   

  58:        // Retriving a single document

  59:        Document getOne = db.getDocument("1");

  60:        System.out.println("Document returned from database : " + getOne.toString());

  61:        getOne = null;

  62:   

  63:        // Delete the document from Database

  64:        getOne = db.getDocument("1");

  65:        db.deleteDocument(getOne);

  66:        count = db.getDocumentCount();

  67:        System.out.println("Total Documents after delete : " + count);

  68:        getOne = null;

  69:   

  70:   

  71:        // cleaning all docs if exist more than one

  72:        ViewResults viewResults = db.getAllDocuments();

  73:        List<Document> docs = viewResults.getResults();

  74:        for(Document doc1 : docs) {

  75:           System.out.println("Document to be deleted : " + doc1.toString());

  76:           db.deleteDocument(doc1);

  77:        }

  78:   

  79:   

  80:        // Delete the database

  81:        mySession.deleteDatabase(dbname);

  82:        listofdb = mySession.getDatabaseNames();

  83:        System.out.println("Number Total Of Databases after delete that created one : " + listofdb.size());

  84:   

  85:     }

  86:  }

 

There is one other way to persist one NoSQL DB is using HTTP:

HttpClient httpclient = new DefaultHttpClient();

HttpGet get = new HttpGet("http://localhost:5984/employeeTest/_all_docs?startkey=%221%22&limit=1");

HttpResponse response = httpclient.execute(get);

 

Or :

HttpGet get = new HttpGet(http://localhost:5984/employeeTest/_all_docs?startkey=%223%22&endkey=%226%22)

Where %22 is used for single cote ( ‘ ).

One last thing, have in mind that CouchDB is not a relational database, so it will just allow querying data based on "key" only, so we cannot retrieve data based on field values from such databases.

This is just a simple example using CouchDB and with one very simple table structure, though for more complex things, as the DB is not relational you will need search using keys and then populate it to create what you might need.

XStream – One example

Here I'll talk about my last experience using the small, however very powerful marshalling/unmarshalling XML framework, the Xstream, you can check it out into http://xstream.codehaus.org/ .
First of all, this framework has a very small API, especially for persistence and it's very powerful and fast to export your objects for one XML file, though to read it you need use JAXP, JAXB, IO or ant other library that read XML code, even the NIO from Java 6 will work.
You can download the binaries from here.
After unzip the file in your choose destination, you will have into the folder this tree, here in my case I'm using the version 1.3.1 :

xstream-1.3.1
	doc	API's and tutorials
	libs	.jars / libs
	LICENSE.txt	Text files
	README.txt	Text files

You will require xstream-[version].jar and xpp3-[version].jar in the classpath. XPP3 is a very fast XML pull-parser implementation. If you do not want to include this dependency, you can use a standard JAXP DOM parser instead.

And here our bind classes, pojos :

ShipTo.java :

public class ShipTo {

    private String name;

    private String address;

    private String city;

    private String country;

    public String getName() {

        return name;

    }

    public void setName(String name) {

        this.name = name;

    }

    public String getAddress() {

        return address;

    }

    public void setAddress(String address) {

        this.address = address;

    }

    public String getCity() {

        return city;

    }

    public void setCity(String city) {

        this.city = city;

    }

    public String getCountry() {

        return country;

    }

    public void setCountry(String country) {

        this.country = country;

    }

} 

 

Item.java :

public class Item {

    private String title;

    private String note;

    private String quantity;

    private String price;

    public String getTitle() {

        return title;

    }

    public void setTitle(String title) {

        this.title = title;

    }

    public String getNote() {

        return note;

    }

    public void setNote(String note) {

        this.note = note;

    }

    public String getQuantity() {

        return quantity;

    }

    public void setQuantity(String quantity) {

        this.quantity = quantity;

    }

    public String getPrice() {

        return price;

    }

    public void setPrice(String price) {

        this.price = price;

    }

} 

 

ShipOrder.java :

public class ShipOrder {

    private String orderid;

    private String orderperson;

    private ShipTo shipto;

    private List<Item> item;

    public String getOrderid() {

        return orderid;

    }

    public void setOrderid(String orderid) {

        this.orderid = orderid;

    }

    public String getOrderperson() {

        return orderperson;

    }

    public void setOrderperson(String orderperson) {

        this.orderperson = orderperson;

    }

    public ShipTo getShipto() {

        return shipto;

    }

    public void setShipto(ShipTo shipto) {

        this.shipto = shipto;

    }

    public List<Item> getItem() {

        return item;

    }

    public void setItem(List<Item> item) {

        this.item = item;

    }

} 

 

Now our class that will serialize to one String as XML and then de-serialize to Objects.

   1:  /*

   2:   * To change this template, choose Tools | Templates

   3:   * and open the template in the editor.

   4:   */

   5:  package xstreamtest;

   6:   

   7:  import com.thoughtworks.xstream.XStream;

   8:  import java.util.ArrayList;

   9:  import java.util.List;

  10:   

  11:  /**

  12:   *

  13:   * @author Klaus

  14:   */

  15:  public class Main {

  16:   

  17:     /**

  18:      * @param args the command line arguments

  19:      */

  20:     public static void main(String[] args) {

  21:        // Getting the Xstream

  22:        XStream xstream = Main.getStream();

  23:   

  24:        // Serializing the object to XML

  25:        ShipTo shipTo = Main.getShipTo();

  26:        List<Item> listOfItems = Main.getItems();

  27:        ShipOrder shipOrder = Main.getShipOrder(shipTo, listOfItems);

  28:   

  29:        // Serializing to the XML into one String, can create file if you want it.

  30:        String shipOrderXml = xstream.toXML(shipOrder);

  31:   

  32:        // Display the XML created

  33:        System.out.println("The XML :");

  34:        System.out.println(shipOrderXml);

  35:   

  36:        // Populating the obj again

  37:        ShipOrder shipOrder1 = (ShipOrder) xstream.fromXML(shipOrderXml);

  38:        ShipTo shipTo1 = shipOrder1.getShipto();

  39:        List<Item> listOfItems1 = shipOrder1.getItem();

  40:   

  41:        System.out.println("-------------------------------");

  42:        System.out.println("- Desirializing to the object -");

  43:        System.out.println("-------------------------------");

  44:        System.out.println("Order Id     : " + shipOrder1.getOrderid());

  45:        System.out.println("Order Person : " + shipOrder1.getOrderperson());

  46:        System.out.println("Ship To      : " + shipOrder1.getShipto().getName());

  47:        System.out.println("Items : ");

  48:        for (Item item : listOfItems1) {

  49:           System.out.println("Title : " + item.getTitle());

  50:           System.out.println("Qtde. : " + item.getQuantity());

  51:           System.out.println("Price : " + item.getPrice());

  52:           System.out.println("Note  : " + item.getNote());

  53:           System.out.println("-----------------------------------------");

  54:        }

  55:     }

  56:   

  57:     public static XStream getStream() {

  58:        XStream xstream = new XStream();

  59:        xstream.alias("shipTo", ShipTo.class);

  60:        xstream.alias("item", Item.class);

  61:        xstream.alias("shipOrder", ShipOrder.class);

  62:        return xstream;

  63:     }

  64:   

  65:     public static List<Item> getItems() {

  66:        List<Item> items = new ArrayList<Item>();

  67:        Item item = new Item();

  68:        item.setTitle("Item 01");

  69:        item.setPrice("10.00");

  70:        item.setQuantity("5");

  71:        item.setNote("This is the first item");

  72:        items.add(item);

  73:   

  74:        item = new Item();

  75:        item.setTitle("Item 02");

  76:        item.setPrice("15.00");

  77:        item.setQuantity("7.50");

  78:        item.setNote("This is the second item");

  79:        items.add(item);

  80:   

  81:        return items;

  82:     }

  83:   

  84:     public static ShipTo getShipTo() {

  85:        ShipTo shipTo = new ShipTo();

  86:        shipTo.setName("Customer 01");

  87:        shipTo.setCity("City 01");

  88:        shipTo.setCountry("country 01");

  89:        shipTo.setAddress("Address 01");

  90:        return shipTo;

  91:     }

  92:   

  93:     public static ShipOrder getShipOrder(ShipTo shipTo, List<Item> items) {

  94:        ShipOrder so = new ShipOrder();

  95:        so.setOrderid("Order number 01");

  96:        so.setOrderperson("Order to Person 01");

  97:        so.setShipto(shipTo);

  98:        so.setItem(items);

  99:        return so;

 100:     }

 101:  }

 

As you can see, is quite easy, serialize as de-serialize the XML with Xstream.

The framework is quite small though powerful and easy to use, make it as one excellent option.

A Few Things To Take In Consideration Before Start One Agile Project

For my experience, that is more than twenty years in IT, and since beginning did try do some sort of Agile (beginning in 1991), when the project wasn’t big and one mix with Waterfall when with Enterprise solutions. Today by experience I’d know that is quite possible develop Enterprise solutions using Agile though a lot of things has changes since when. Especially with IDEs that 15-20 years ago where nearly inexistent, my preferred editor was the NE the good and old Norton Editor in DOS and  into Unix and Xenix  (if you do remember this last one, you should be considered a Dinosaur from IT, just like me!).

Must have more items to thing about it, though those ones for me are like the basics ones and will explain each one here. I know that some people may will say, “Aaaahhhh the number one isn’t true, the number X isn’t too, or even that this guy doesn’t understand about Agile”, however the truth is that I’d never have delivered one project with delays and the conclusion you should be take for yourself.

One thing for sure I’m not. I’m note that guy who want use EJB X, or Spring and bloat the system with patterns when isn’t needed. I like to talk technical language, though when talking with Juniors, is better you do use some examples with some sort of analogy that they will understand quite better. Be technical is good for who can understand, for those ones that are just starting, is one little bit scaring you be too technical. I think that they should have time to get there, and the first thing in one project that they need to know is, ‘know’ what they have to implement, and that they have one team leader that will be a team leader in fact. Teaching and explaining things when they need, they must feel that you are like a friend, that they can ask you anything about the project, language, patterns or correlated things without fear to be ridiculous. Doing this for sure they will have one better production and will close the gap about the system and those technologies involved.

You for sure should manage the code creation and have some patterns or conventions about it, and let your team familiarized with it as soon as the project starts. Comments exist and make then use it, explanations are never enough. This will help a lot when someone have to enhance or update the system.

If enhancing/updating or even maintaining one old system, to just create new classes and methods when strictly necessary, even that the code is the real bad quality. If you have time to re-write it, cool do it, if not, keep with the strict that is delivery the system working well. Doesn’t creating unnecessary classes and methods believe or not make the production go fast and doesn’t change the system structure, even being a bas one. Because, may be the next one to ‘touch’ in the system, will be the creator or someone that has to read follow all steps that you did to understand it and if you starting changing it, for sure the system will become a Frankstein and for sure will be harder to maintain.

One thing that I always put as top priorities are those two ROI and Quality, that means delivery good code and working of course, and that match with what the client expect and that has paid for.

Now for you that doesn’t have enough experience making run one Agile project I do enumerated a few things that should help you. 

1 . Just juniors will never able to work with/implement it.

   Nothing against them though if you are the only Senior in the project, be ready to do a lot of extra hours and have to re-write a lot of code, not all juniors are equals, but most of then take long time to delivery something. So  if you want make it go right since beginning mix at least one rate of 1 to 1, that means one with experience and one junior. If you can’t do that, don’t put the project and yourself in risk, go for some sort of mix, or something that is there before, keep one backup that can help you when you do need at most. Most of juniors must be guided, due this they love RUP and any Waterfall, because take off their hands the responsibility.

 

2 . Only years and having worked in several projects will give you the expertise to make run one successful Agile project, the same way one senior struts developer will never be a good system architect.

    This means that rush to be something that you are not ready to be, will just make you frustrated, saying that Agile does not work, like I’d saw a lot in several places on internet. If you don’t have any expertise with it, try create pilot projects, easy things to implement and try keep some sort of back up in whatever methodology was there before. With time you will be able to start spotting the issues and the potentials source of problems and delivery have solutions for it. Usually even reading a lot you will struggle in your first projects. If at anytime you don’t feel secure, raise the flag and call for help, it’s better you be honest with yourself and with is paying you than wast time, money and probably  lost your job. This also means that if someone ask you something that you don’t know, keep the head up and say that you will research about it and soon will let he/she know about it.

 

3 . The team must have the 'I can do it' attitude and be a experienced and organized team (this is one of the most hard to put together).

     This is a must in Agile projects, if any person of your team is showing you be one little bit lazy with lack of this attitude, talk with him a few times, though don’t expend too much time if you feel that he/she doesn’t want to change, take he/she off from the project fast. You can see this for the meetings, usually in the beginning of the project I do those very short meeting every day, after some time when they are familiarized with each other, I do go for one maximum of 3 meeting weekly, though most of projects one or two weekly are more than enough.

 

4 . Just follow what the book does not help if you don't have a good experience.

     I think that doesn’t need explain too much this right. If you don’t have enough experience and just try follow one book, you are putting a lot of things in risk, and one these include you, that depending of where you are trying implement it, you can kill your career before even started.

 

5 . Does not exist one standard way to make run Agile projects though just conventions, good practices, nor formula.

   As do wrote here before, your experience and being successful in find the potential problems that will make you run with success one Agile project.
   

 

6 . Use TDD with average team, will not help, you'll need a good team.

    Try follow it, if not TDD, test a lot your system in all possible ways, fail gracefully if inevitable, pay a lot attention with those nulls pointer exceptions, test those business rules up to exhaustion,don’t be lazy and try ask for some team mate test your code, when you do test other code. Check the data’s, create Unit tests, for everything that you had created and if possible one that will perform all tasks one full process. Try keep more realistic possible those data’s that will help you spot any issue easier than with auto generated data’s. After has tested for data’s do loading test and performance tests. Those ones will give you one good idea of how many connections your system can handle, the maximum before fail, and memory used.

 

7 . Need have access to the people that have the knowledge about what is being developed, no access means lack of information and fail.

    Without access to those ones that have the knowledge about what is being developed is waste time with Agile, don’t do it, because you will create one system that just get close, or not of what your client need and with quite less documentation, that means if he/she want make those changes to make the system match with what they expect, will be quite hard, and even you not being there anymore they will blame you for sure. My opinion is… “Always leave leaving the door open for you”. No ones can have the next day as granted right!

 

8 . At last one person of your team must have expertise in all project phases, like from make the interviews up to deploy in production environment, and know most of configurations for the environment to be used, just code does not help.

    This is very important, if you don’t have this expertise don’t do it or find someone that can really help you when you need. Remember Murphy’s Law… “If something might go wrong… it will go wrong”, and probably this will happen when you less expected.

 

Because this, isn’t so easy implement agile in medium to big projects and it's expensive for the company have several good guys working with it, and means that after project done, they will need keep a few just to enhance the system after, or to do the maintenance.
Before start one project you should know at least these points before to decide for what methodology you should go.
Of course there are successful histories in all methodologies, though doesn't mean that it will work all the time, and one very important tip for you. If you fail, please keep the head in place, keep calm, and do the analysis of what went wrong and why, to do not repeat it next time.

These are a few tips that I’d use very often and hope that it help you when you’d need it.