Adding JSON Documents into GemFire Cache

The JSONFormatter API allows you to put JSON formatted documents into regions and retrieve them later by storing the documents internally as PdxInstances. vFabric GemFire now supports the use of JSON formatted documents natively. When you add a JSON document to a GemFire cache, you call the JSONFormatter APIs to transform them into the PDX format (as a PdxInstance), which enables GemFire to understand the JSON document at a field level.
 
In terms of querying and indexing, because the documents are stored internally as PDX, applications can index on any field contained inside the JSON document including any nested field (within JSON objects or JSON arrays.) Any queries run on these stored documents will return PdxInstances as results. To update a JSON document stored in GemFire, you can execute a function on the PdxInstance.

You can then use the JSONFormatter to convert the PdxInstance results back into the JSON document.

Here is a simple example.

1. server side cache.xml file, in this demo we just start up one cache server

cache.xml

  
<?xml version="1.0"?>
<!DOCTYPE cache PUBLIC
    "-//GemStone Systems, Inc.//GemFire Declarative Caching 7.0//EN"
    "http://www.gemstone.com/dtd/cache7_0.dtd">

<cache>
    <cache-server port="40404"/>
    <region name="jsonregion">
       <region-attributes refid="REPLICATE" />
    </region>
</cache>

2. Client side cache.xml

client.xml

  
<?xml version="1.0"?>
<!DOCTYPE client-cache PUBLIC
    "-//GemStone Systems, Inc.//GemFire Declarative Caching 7.0//EN"
    "http://www.gemstone.com/dtd/cache7_0.dtd">

<!--
  | Client.xml
  |
  | Configures a region as a client region in a client/server cache. The 
  | region's pool connects to a cacheserver listening on port 40404.
 -->
<client-cache>
  <pool name="client" subscription-enabled="true">
    <server host="localhost" port="40404" />
  </pool>

  <region name="jsonregion">
    <region-attributes refid="PROXY">      
    </region-attributes>
  </region>
</client-cache>  

3. Java class to insert some JSON formatted documents into the cache then how to query them back using a field from the JSON document itself.

JSONGemFireClient.java

  
package vmware.au.gemfire7.json.client;

import java.util.List;

import org.json.simple.JSONObject;

import com.gemstone.gemfire.cache.Region;
import com.gemstone.gemfire.cache.client.ClientCache;
import com.gemstone.gemfire.cache.client.ClientCacheFactory;
import com.gemstone.gemfire.cache.query.SelectResults;
import com.gemstone.gemfire.pdx.JSONFormatter;
import com.gemstone.gemfire.pdx.PdxInstance;

public class JSONGemFireClient 
{

   public static final String REGION_NAME = "jsonregion";
   public ClientCache cache = null;
   
   public JSONGemFireClient()
   {
    cache = new ClientCacheFactory()
          .set("name", "JSONClient")
          .set("cache-xml-file", "xml/client.xml")
          .create();   
   }
 
   @SuppressWarnings("unchecked")
   public void run() throws Exception
   {
    JSONObject obj = null;
    
    System.out.println("Connecting to the distributed system and creating the cache.");
      
    // Get the exampleRegion
    Region<String, PdxInstance> jsonregion = cache.getRegion(REGION_NAME);
    System.out.println("Example region \"" + jsonregion.getFullPath() + "\" created in cache.");
    
    // add 5 entries with age = 30
     
    for (int i = 1; i <= 5; i++)
    {
      obj = new JSONObject();
      
      obj.put("name", String.format("Person%s", i));
      obj.put("age", 30);
     
      jsonregion.put(String.valueOf(i), JSONFormatter.fromJSON(obj.toJSONString()));
    }
      
    // add 5 entries with age = 20
    for (int i = 6; i <= 10; i++)
    {
      obj = new JSONObject();
      
      obj.put("name", String.format("Person%s", i));
      obj.put("age", 20);
     
      jsonregion.put(String.valueOf(i), JSONFormatter.fromJSON(obj.toJSONString()));
    }
    
    // Query region
    SelectResults<PdxInstance> sr = jsonregion.query("age = 30");
    
    System.out.println("Number of entries where age = 30 is -> " + sr.size());
    
    List<PdxInstance> entries = sr.asList();
    for (PdxInstance val: entries)
    {
     System.out.println("\n** JSON data ** ");
     System.out.println("Name = " + val.getField("name"));
     System.out.println("Full JSON data -> \n" + JSONFormatter.toJSON(val));
    }
    
    cache.close();
   }
 
   /**
    * @param args
    */
   public static void main(String[] args) 
   {
    // TODO Auto-generated method stub
    JSONGemFireClient test = new JSONGemFireClient();
    
    try 
    {
   test.run();
    } 
    catch (Exception e) 
    {
   // TODO Auto-generated catch block
   e.printStackTrace();
    }
   }

}

4. Run the java class above and verify output as follows

....
[info 2013/02/13 09:41:15.464 EST

tid=0x1] Defining: PdxType[
      id=1, name=__GEMFIRE_JSON, fields=[
          age:byte:0:idx0(relativeOffset)=0:idx1(vlfOffsetIndex)=0
          name:String:1:idx0(relativeOffset)=1:idx1(vlfOffsetIndex)=-1]]
Number of entries where age = 30 is -> 5

** JSON data **
Name = Person1
Full JSON data ->
{
  "age" : 30,
  "name" : "Person1"
}

** JSON data **
Name = Person4
Full JSON data ->
{
  "age" : 30,
  "name" : "Person4"
}

** JSON data **
Name = Person2
Full JSON data ->
{
  "age" : 30,
  "name" : "Person2"
}

** JSON data **
Name = Person5
Full JSON data ->
{
  "age" : 30,
  "name" : "Person5"
}

** JSON data **
Name = Person3
Full JSON data ->
{
  "age" : 30,
  "name" : "Person3"
}
....

5. Finally use GFSH to query the region itself and you wouldn't even know your using JSON stored documents and would rely on using JSONFormatter  to convert the data back into JSON document.

  
gfsh>connect --jmx-manager=localhost[1099]
Connecting to Manager at [host=localhost, port=1099] ..
Successfully connected to: [host=localhost, port=1099]

gfsh>list regions;
List of regions
---------------
jsonregion

gfsh>query --query="select * from /jsonregion";

Result     : true
startCount : 0
endCount   : 20
Rows       : 10

age | name
--- | --------
30  | Person1
30  | Person4
20  | Person8
20  | Person10
20  | Person6
30  | Person2
30  | Person5
20  | Person9
20  | Person7
30  | Person3

NEXT_STEP_NAME : END 

More information can be found on the link below.

http://pubs.vmware.com/vfabricNoSuite/index.jsp?topic=/com.vmware.vfabric.gemfire.7.0/developing/data_serialization/jsonformatter_pdxinstances.html