Pagination in mobile development

27.07.201912 Min Read — In programming

#Intro

I decided to write this article since I have been implementing pagination in iOS numerous of times. And all these implementations were different, and it was hard to reuse them. So the next time when I'd been implementing pagination for the mobile app, I decided to create a generic interface for working with it and share it with everyone.

If you already know what does term pagination means and how it looks like in mobile apps, you can switch right to the Technical part.

#What is pagination

Imagine some E-commerce or Real Estate platform with tens or even hundreds of thousands of listing entries at their database.

Loading all of them at once seems to be a not really reasonable idea.

Even if you'll try do that, your browser or a mobile app most likely become unresponsive for noticeable amount of time or just stuck until the process will be killed.

And opposite, loading data in portions, page by page, upon user requests, won't cause bad user experience mentioned above.

So what is pagination?

Wikipedia says:

Pagination, also known as Paging, is the process of dividing a document into discrete pages, either electronic pages or printed pages.

source: Wikipedia

But in practice definition of pagination might vary depending on a purpose. Being more concrete from the mobile development perspective, I would formulate it in the following way:

Pagination is a mechanism, which allows fetching a collection of similar elements by the given range or by page number from some database.

#When to use it

Why do you need it on your mobile app?

  • it allows to reduce data loading time, therefore provides smoother UX and reduces server load. 
  • it just saves battery and network traffic.

It’s fair to say that pagination is not just “nice to have”, it is a required technique in the mobile app or a web app development, if it has to be dealing with a significant amount of data.

#Pagination Designs

Pagination UI has two most common variations:

  • Continuous page (aka Infinite Scrolling)
  • Navigation buttons on the bottom of each page

In mobile apps, it's become a standard to use Continuous Page.

Continuous page example:

While in web design both versions have taken root there - Continuous page and Page Navigation buttons on the bottom of each page.

Navigation buttons example:

In the case of Navigation buttons to load the next page, explicit user action is required, while for the second case data loading triggered indirectly, whenever reaching the bottom of the screen's content.

For a brief pagination designs overview, you can check out this link at dribble Dribble.

#Technical side

Pagination on the client side consists of two major parts.

The first part belongs to UI logic, to observe and notify about the moment when to request new data.

The second part is to transform UI event into a request object, retrieve and process the data for the next page. 

Below you can see an overall scheme explaining how it works.

Whenever corresponding user action is happening, the UI layer is triggering Pagination Controller API to load "more" items. In its turn Pagination Controller converts this event to the specific URL and performs network request (it could also be a request to local storage), and by the request's completion handles result and notifies UI layer to update accordingly.

In this article, I'll focus on the details of the second part.

#Querying the data

As an input query, it could be parameters either like offset and limit or just plain links, like first, last, prev and next if your backend supports JSON:API.

In the offset and limit case, it is a client's job to build a URL with proper parameters.     

Here is how can it look like:

GET /articles?limit=10&offset=20

According to this link, the server should skip the first 20 articles and return the next 10 following articles stored at its database.

Where in the JSON:API case, the client app trusts and uses URL, generated and provided by the server side.

Here is JSON:API URL example:

GET /articles?page[number]=2&page[size]=10

Since in my experience, client apps mostly are dealing with offset and limit, therefore I want to stop at this variation and explain how client apps can interact with such API. 

#Offset and limit

For the sake of better understanding how offset and limit used to query the data I've created following code snippet:

let items = [0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20]
let offset = 9
let limit = 8
let queryResult = Array(items[offset..<(offset + limit)]) // [9,10,11,12,13,14,15,16]

Where queryResult variable here is the simulation of our imaginary network response.

Adjusting the limit you are changing a number of items per "page", and modifying an offset (in some API variations also called skip) you are "jumping" between pages.

Note: Code snippet from above is very simplified just to show how pagination API works on client and server sides, and it doesn’t covers the edge case situations such as querying items by not existing range.

#Coding Part 

Let's start from declaring protocol.

Say we want to retrieve listing items from a Real Estate platform.

Since our main goal to retrieve paginated data, let’s declare the following interface:

protocol PaginationSupportable {
    associatedtype PaginatedItem
    var dataSource: [PaginatedItem] { get set }
}

PaginatedItem here is the placeholder type name. When you extend your type to conform this protocol, the compiler will ask you to specify PaginatedItem type explicitly.

And dataSource attribute will be used for storing the data retrieved from the server.

So our hypothetical Real Estate listings data provider type will look like this:

struct ListingItem {
   let id: String
   let title: String
}

class ListingItemsDataProvider: PaginationSupportable {
   typealias PaginatedItem = ListingItem
   var dataSource: [ListingItem] = []
}

The line typealias PaginatedItem = ListingItem here is not necessary, as the compiler can infer the associated type from dataSource property, so it will be omitted at the furthering code samples.                                 

Then we need to get knowledge for generating request objects, therefore we’re adding offset and limit into our protocol.

The updated interface will look like this:

protocol PaginationSupportable {
    associatedtype PaginatedItem
    var dataSource: [PaginatedItem] { get set }
    var limit: Int { get }
    var offset: Int { get }
}

The information which can be provided by the type conforming to this protocol is already enough to build a query for the network API. So let's reflect this to our protocol:

protocol PaginationSupportable {
    associatedtype PaginatedItem
    associatedtype Request
    var dataSource: [PaginatedItem] { get set }
    var limit: Int { get }
    var offset: Int { get }
    var request: Request { get }
}

Request type here can be a URLRequest object or any custom type according to your application network layer's API. For simplicity, we will use a [String: Any] dictionary as a Request type .

Let’s assume our page size is equal to 8 items per page, where we want to show Continuous Page. After updating ListingItemsDataProvider accordingly, it will look like this:

class ListingItemsDataProvider: PaginationSupportable {
    var dataSource: [ListingItem] = []
    var limit: Int { return 8 }
    var offset: Int { return dataSource.count }
    var request: [String : Any] = [:]
}

Note: If you want to refresh already downloaded elements, then you have to adjust an offset parameter or process newly downloaded elements accordingly.

From this moment we can start integrating with the network API.

But in terms of this article for this purpose we will use optimized snippet from Offset and Limit part.

Here it is:

struct Server {
    var mockData: [ListingItem] {
        return [Int](1...100).map { ListingItem(id: "\($0)", title: "Item \($0)) }
    }

    func queryData(request: [String : Any], completion: @escaping (([ListingItem])->Void)) {
        let limit = request["limit"] as! Int
        let offset = request["offset"] as! Int
        let data = Array(mockData[offset..<(offset+limit)])
	    DispatchQueue.global(qos: .background).asyncAfter(deadline: .now() + 1) {
            completion(data)
        }
    }
}

You may think about queryData function as a wrapper of the backend endpoint. To make it feel more realistic I’ve added delay in 1 second, calling GCD's asyncAfter method.

Then let’s extend PaginationSupportable protocol with trigger function we will use for retrieving paginated items.

protocol PaginationSupportable {
    associatedtype PaginatedItem
    associatedtype Request
    var dataSource: [PaginatedItem] { get set }
    var limit: Int { get }
    var offset: Int { get }
    var request: Request { get }
    func queryData(request: Request, completion: @escaping (([PaginatedItem])->Void))
}

And update ListingItemsDataProvider again, but for this time, connecting it with mock network endpoint:

class ListingItemsDataProvider: PaginationSupportable {
    var dataSource: [ListingItem] = []
    var limit: Int { return 8 }
    var offset: Int { return dataSource.count }
    var request: [String : Any] {
        return ["limit" : limit,
                "offset" : offset]
    }

    private let server = Server()

    func queryData(request: [String : Any], completion: @escaping (([ListingItem]) -> Void)) {
        server.queryData(request: request, completion: completion)
    }
}

And finally we can see our first result:

let listingsDataProvider = ListingItemsDataProvider()
listingsDataProvider.queryData(request: listingsDataProvider.request) { items in
    print(items.map { $0.title })
    // prints ["Item 1", "Item 2", "Item 3", "Item 4", "Item 5", "Item 6", "Item 7", "Item 8"]
}

Not bad already, but we don’t store the data yet, therefore offset variable won't be changed, and query result will keep remaining the same. So let’s go back to ListingItemsDataProvider and update it again.

class ListingItemsDataProvider: PaginationSupportable {
    var dataSource: [ListingItem] = []
    var limit: Int { return 8 }
    var offset: Int { return dataSource.count }
    var request: [String : Any] {
        return ["limit" : limit,
                "offset" : offset]
    }

    private let server = Server()

    func queryData(request: [String : Any], completion: @escaping (([ListingItem]) -> Void)) {
        let old = self.dataSource
        server.queryData(request: request) { [weak self] new in
            self?.dataSource = old + new
            completion(new)
        }
    }
}

Also for the convenience let’s add queryMore func as an extension of PaginationSupportable protocol, which will be loading the next page data again and again, whenever its get called.

extension PaginationSupportable {									
    func queryMore(completion: @escaping (([PaginatedItem])->Void)) {			
        queryData(request: request, completion: completion)					
    }														
}														

Its benefit is just a shortcut for calling the next page, and also to get rid of redundancy such as providing request parameter listingsDataProvider.request which is already belongs to a called instance listingsDataProvider.

Now we have a full mechanism, ready for querying listing items. Whenever a user reaches the bottom of a Continuous Page, an external trigger should be calling queryMore function.  

As I mentioned earlier, we will focus on the technical side, so UI part left to a reader. Feel free to post your implementations in comments.

But for our example, we will use vanilla GCD.

let listingsDataProvider = ListingItemsDataProvider()								
																
listingsDataProvider.queryMore { data in										
    print("page-1: \(data.map { $0.title })")									
    listingsDataProvider.queryMore { data in									
        print("page-2: \(data.map { $0.title })")									
        listingsDataProvider.queryMore { data in									
            print("page-3: \(data.map { $0.title })")								
        }															
    }																
}																
																
/* prints:															
page-1: ["Item 1", "Item 2", "Item 3", "Item 4", "Item 5", "Item 6", "Item 7", "Item 8"]		
page-2: ["Item 9", "Item 10", "Item 11", "Item 12", "Item 13", "Item 14", "Item 15", "Item 16"]	
page-3: ["Item 17", "Item 18", "Item 19", "Item 20", "Item 21", "Item 22", "Item 23", "Item 24"]	
*/	

- Yay, it is almost done!

- Wait… almost?!?!

- Yeah, there’s one thing left. What happens if the server returns the data, which client side has been already cached?

According to current logic, there's no differentiation between old and newly downloaded items. New ones, simply being added to the end of the list:

 self?.dataSource = old + new

So there’s high chances, that our hypothetical Continuous Page can show more than one similar listing items.

But nothing prevents us from implementing a Diffable protocol that can be used for finding duplicates and excluding similar or identical content. Let’s see how it works.

At first we declare Diffable protocol itself:

protocol Diffable {				
    var uniqueId: String { get }		
}							

Then we add Diffable constraint to PaginatedItem:

protocol PaginationSupportable {	
    associatedtype PaginatedItem: Diffable	
    associatedtype Request 
    var dataSource: [PaginatedItem] { get set }		
    var limit: Int { get }
    var offset: Int { get }
    var request: Request { get }
    func queryData(request: Request, completion: @escaping (([PaginatedItem])->Void))
}				 

And implement this protocol on ListingItem:

extension ListingItem: Diffable {					
    var uniqueId: String {						
        return id								 
    }										
}										

Now ListingItemsDataProvider has enough knowledge on how to detect duplicates and replace existing items with new ones.

Here is brief code example how it works:

let query1 = [(1, "a"),(2, "a"),(3, "a"),(4, "a"),(5, "a"),(6, "a"),(7, "a")]                
let query2 = [(6, "b"),(7, "b"),(8, "b"),(9, "b")]                                   
var merged = query1.filter { old in !query2.contains(where: { old.0 == $0.0 }) } + query2         

// old:                                                          
// [(1, "a"), (2, "a"), (3, "a"), (4, "a"), (5, "a")]   

// replaced:                                                          
// [(6, "a"), (7, "a")] to [(6, "b"), (7, "b")]   

// new:                                                          
// [(8, "b"), (9, "b")]   

Let’s implement same logic at ListingItemsDataProvider. First we will add processData function at ListingItemsDataProvider:

func processData(data new: [PaginatedItem]) -> [PaginatedItem] {    
        let merged = dataSource.filter { old in !new.contains(where: { old.uniqueId == $0.uniqueId }) } + new                                                 
        return merged                                                   
}          

And second, we will update queryData function in a following way:

func queryData(request: [String : Any], completion: @escaping (([ListingItem]) -> Void)) {  
        server.queryData(request: request) { [weak self] new in                      
            guard let self = self else { return }                               
            self.dataSource = self.processData(data: new)                       
            completion(new)                                         
        }                                                       
}

Or alternatively, we can make processData func to be a part of PaginationSupportable and add its default implementation:

protocol PaginationSupportable {                                       
    associatedtype PaginatedItem: Diffable                                  
    associatedtype Request                                           
    var dataSource: [PaginatedItem] { get set }                             
    var limit: Int { get }                                          
    var offset: Int { get }                                         
    var request: Request { get }                                        
    func queryData(request: Request, completion: @escaping (([PaginatedItem])->Void))       
    func processData(data new: [PaginatedItem]) -> [PaginatedItem]                  
}                                                           
                                                            
extension PaginationSupportable {                                       
    func processData(data new: [PaginatedItem]) -> [PaginatedItem] {                
        let merged = dataSource.filter { old in !new.contains(where: { old.uniqueId == $0.uniqueId }) } + new                                             
        return merged                                               
    }                                                           
}                                                           

So for every type which conforms to PaginationSupportable, data will be processed in a similar way, unless it implements its own custom processData logic.

And finally, that’s it!

Pagination mechanism is ready to use, it just remains to connect it to UI.

Below is the complete implementation: 

protocol Diffable {                                                    
    var uniqueId: String { get }                                            
}                                                               
                                                                
protocol PaginationSupportable {                                            
    associatedtype PaginatedItem: Diffable                                      
    associatedtype Request                                              
    var dataSource: [PaginatedItem] { get set }                                 
    var limit: Int { get }                                              
    var offset: Int { get }                                             
    var request: Request { get }                                            
    func queryData(request: Request, completion: @escaping (([PaginatedItem])->Void))           
    func processData(data new: [PaginatedItem]) -> [PaginatedItem]                      
}                                                               
                                                                
extension PaginationSupportable {                                           
    func processData(data new: [PaginatedItem]) -> [PaginatedItem] {                    
        let merged = dataSource.filter { old in !new.contains(where: { old.uniqueId ==      $0.uniqueId }) } + new                                                  
        return merged                                                   
    }                                                               
                                                                    
    func queryMore(completion: @escaping (([PaginatedItem])->Void)) {                   
        queryData(request: request, completion: completion)                         
    }                                                               
}   															

And usage example:

struct ListingItem: Diffable {                                             
    let id: String                                                      
    let title: String                                                   
    var uniqueId: String {                                              
        return id                                                       
    }                                                               
}                                                               
                                                                
struct Server {                                                     
    var mockData: [ListingItem] {                                           
        return [Int](1...100).map { ListingItem(id: "\($0)", title: "Item \($0)") }         
    }                                                               
                                                                    
    func queryData(request: [String : Any], completion: @escaping (([ListingItem])->Void)) {        
        let limit = request["limit"] as! Int                                    
        let offset = request["offset"] as! Int                                  
        let data = Array(mockData[offset..<(offset+limit)])                         
        DispatchQueue.global(qos: .background).asyncAfter(deadline: .now() + 1) {           
            completion(data)                                                
        }                                                           
    }                                                               
}                                                               
                                                                
class ListingItemsDataProvider: PaginationSupportable {                             
    private let server = Server()                                           
    var dataSource: [ListingItem] = []                                      
    var limit: Int { return 8 }                                         
    var offset: Int { return dataSource.count }                                 
    var request: [String : Any] {                                           
        return ["limit" : limit,                                            
                "offset" : offset]                                          
    }                                                               
                                                                    
    func queryData(request: [String : Any], completion: @escaping (([ListingItem]) -> Void)) {  
        server.queryData(request: request) { [weak self] new in                     
            guard let self = self else { return }                                   
            self.dataSource = self.processData(data: new)                           
            completion(new)                                             
        }                                                           
    }                                                               
}                                                               
                                                                
let listingsDataProvider = ListingItemsDataProvider()                               
                                                                
listingsDataProvider.queryMore { data in                                        
    print("page-1: \(data.map { $0.title })")                                   
    listingsDataProvider.queryMore { data in                                    
        print("page-2: \(data.map { $0.title })")                                   
        listingsDataProvider.queryMore { data in                                    
            print("page-3: \(data.map { $0.title })")                               
        }                                                           
    }                                                               
} 
 
// prints:
// page-1: ["Item 1", "Item 2", "Item 3", "Item 4", "Item 5", "Item 6", "Item 7", "Item 8"]
// page-2: ["Item 9", "Item 10", "Item 11", "Item 12", "Item 13", "Item 14", "Item 15", "Item 16"]
// page-3: ["Item 17", "Item 18", "Item 19", "Item 20", "Item 21", "Item 22", "Item 23", "Item 24"]

#Conclusion

In this article, you’ve learned: what is pagination, when to use it, how pagination UI looks like in the web and mobile apps, and how to implement it technically

I hope you enjoyed reading it and learned something new. Feel free to post your thoughts in the comments.

And stay tuned!