Displaying Huge Amount of Data

Issue of Displaying Huge Amount of Data

• time to query all data is unbearable long at one time.
  • MVVM. NotifyChange a property of ListModel triiggers high-cost re-query of all data which might lead to low performance.
• data size might change for concurrent modifications when viewing.

Solution : Query One Page A Time

implement a custom ListModel. It only queries a page size of data during one request execution and store it as a cache.

package org.zkoss.reference.developer.mvvm.advance;

import java.util.*;
import org.zkoss.reference.developer.mvvm.advance.domain.*;
import org.zkoss.zk.ui.*;
import org.zkoss.zul.AbstractListModel;

public class LivePersonListModel extends AbstractListModel<Person>{

	private static final long serialVersionUID = -7982684413905984053L;
	
	private PersonDao personDao;
	private int pageSize = 10;
	private final String CACHE_KEY= LivePersonListModel.class+"_cache";
	
	public LivePersonListModel(PersonDao personDao){
		this.personDao = personDao;
	}
	
	public Person getElementAt(int index) {
		Map<Integer, Person> cache = getCache();

		Person targetPerson = cache.get(index);
		if (targetPerson == null){
			//if cache doesn't contain target object, query a page size of data starting from the index
			List<Person> pageResult = personDao.findAll(index, pageSize);
			int indexKey = index;
			for (Person o : pageResult ){
				cache.put(indexKey, o);
				indexKey++;
			}
		}else{
			return targetPerson;
		}

		//get the target after query from database
		targetPerson = cache.get(index);
		if (targetPerson == null){
			//if we still cannot find the target object from database, there is inconsistency between memory and the database
			throw new RuntimeException("Element at index "+index+" cannot be found in the database.");
		}else{
			return targetPerson;
		}
	}
	
	private Map<Integer, Person> getCache(){
		Execution execution = Executions.getCurrent();
		//we only reuse this cache in one execution to avoid accessing detached objects.
		//our filter opens a session during a HTTP request
		Map<Integer, Person> cache = (Map)execution.getAttribute(CACHE_KEY);
		if (cache == null){
			cache = new HashMap<Integer, Person>();
			execution.setAttribute(CACHE_KEY, cache);
		}
		return cache;
	}	

	public int getSize() {
		return personDao.findAllSize();
	}


	public int getPageSize() {
		return pageSize;
	}

	public void setPageSize(int pageSize) {
		this.pageSize = pageSize;
	}
}

It works fine under paging or default mold.

Solution : Switch and Update

When you browse a group of data in a database, other users could also add, modify, or delete the same group of data. The total size of this group of data might change during your browsing. We want displaying data to be closer to database's current status, one proper timing is switching pages.