Pagination, Filtering, and Sorting

# CHAPTER 12

Pagination, Filtering, and Sorting #

1. Introduction #

2. Learning Objectives #

• Understand the necessity of chunking data collections.

• Distinguish between Offset Pagination and Cursor Pagination.

• Implement Query Parameters for filtering data.

• Implement Sorting mechanisms.

• Construct comprehensive JSON metadata for paginated responses.

3. Beginner-Friendly Explanation #

• Without REST constraints: The warehouse dumps 10,000 shirts on your front lawn and says, "Find it." (Server Crash).

• Filtering: You ask the warehouse, "Only show me *Blue* shirts."

• Sorting: You ask, "Sort them from *Lowest Price to Highest Price*."

• Pagination: You say, "Show me exactly *10 shirts per page*, and right now, I only want to see *Page 1*."

In REST APIs, we use Query Parameters (the part of the URL after the ?) to pass these instructions to the warehouse (The Database).

4. Query Parameters (The Syntax) #

GET /api/products?category=electronics&sort=price_asc&page=2&limit=20

5. Filtering Data #

Endpoint: GET /api/users?role=admin&status=active

Implementation (Express.js):

app.get('/api/users', async (req, res) => {
    // req.query automatically parses the URL parameters!
    const { role, status } = req.query;
    
    // Build a dynamic database query
    let queryOptions = {};
    if (role) queryOptions.role = role;       // { role: "admin" }
    if (status) queryOptions.status = status; // { status: "active" }

    // Execute query: SELECT * FROM users WHERE role='admin' AND status='active'
    const users = await database.find(queryOptions);
    res.json({ data: users });
});

6. Sorting Data #

Endpoint: GET /api/articles?sort=-created_at (Newest first)

Implementation:

const sortParam = req.query.sort;
let sortOption = {}; // Mongoose syntax example

if (sortParam) {
    if (sortParam.startsWith('-')) {
        // Remove the '-' and sort descending (-1)
        sortOption[sortParam.substring(1)] = -1; 
    } else {
        sortOption[sortParam] = 1; // Sort ascending (1)
    }
}
// db.articles.find().sort(sortOption)

7. Pagination: Offset vs Cursor #

Method 1: Offset/Limit Pagination (Standard) The client requests a specific Page Number and a Limit (Items per page).

• GET /api/posts?page=3&limit=10

• *Database Logic:* Skip the first 20 records (Offset), and take the next 10 (Limit).

• *Pros:* Easy to implement. Allows users to jump straight to Page 50.

• *Cons:* Terrible performance on massive tables (skipping 1,000,000 records takes time). Can return duplicate data if new records are added while the user clicks to the next page.

Method 2: Cursor Pagination (Advanced/Infinite Scroll) The client sends the ID of the last item they saw. The database fetches items *after* that ID.

• GET /api/posts?after_id=945&limit=10

• *Database Logic:* SELECT * FROM posts WHERE id > 945 LIMIT 10

• *Pros:* Lightning fast, regardless of table size. Never returns duplicates. Perfect for Twitter/Instagram-style infinite scrolling.

• *Cons:* You cannot jump to Page 50. You must click "Next" sequentially.

8. Designing the Paginated Response Structure #

The Standard Paginated JSON Response:

{
  "status": "success",
  "data": [
    { "id": 1, "title": "Post 1" },
    { "id": 2, "title": "Post 2" }
  ],
  "meta": {
    "total_records": 150,
    "current_page": 1,
    "total_pages": 15,
    "limit": 10,
    "has_next_page": true,
    "has_prev_page": false
  }
}

9. Best Practices #

• Enforce Default and Maximum Limits: Never trust the client. If a client requests ?limit=1000000, they will crash your server. Always define a strict maximum in your code (e.g., const limit = Math.min(req.query.limit || 20, 100);). This ensures the API defaults to 20 items, and physically cannot return more than 100 items per request, regardless of what the URL says.

10. Common Mistakes #

• Complex Filtering in URLs: If your filtering requirements become incredibly complex (e.g., WHERE (age > 20 AND age < 30) OR (role = admin)), trying to express this logic in a URL Query String becomes a messy nightmare. For highly complex queries, it is acceptable in modern API design to bend REST rules and use a POST /api/users/search endpoint, placing the complex query logic cleanly inside a JSON Request Body.

11. Exercises #

1. 1. Look at this URL: GET /api/movies?genre=scifi&sort=-rating&page=2&limit=5. Explain exactly what this API request is asking the server to do.

12. Coding Challenges #

• Challenge: You are writing Node.js logic to calculate the totalpages metadata field for a paginated response. The database tells you there are totalrecords = 53. The client requested limit = 10. Write a one-line Javascript math equation using Math.ceil() to calculate exactly how many pages exist.

13. MCQs with Answers #

14. Interview Questions #

• Q: Compare Offset/Limit Pagination with Cursor-Based Pagination. In what specific frontend UX scenario (e.g., a data table vs. a social media feed) is Cursor Pagination drastically superior, and why?

• Q: Walk me through the JSON response structure of a paginated API endpoint. Why is it necessary to separate the actual data array from the metadata block?

15. FAQs #

16. Summary #

17. Next Chapter Recommendation #