Migration Guide: v1 to v2

Overview

This guide helps you migrate from the legacy AVERT Imagery API (v1) to the new unified API v2. The migration involves updating endpoints, parameter names, and response handling.

API v1 will continue to work but is no longer actively maintained. We strongly recommend migrating to v2 for better features and ongoing support.

Why Migrate?

Unified API

Single endpoint replaces multiple endpoints

Better Filtering

New atmospheric and temporal filters

Smart Downloads

Get estimates before downloading

Improved Pagination

Comprehensive pagination metadata

Breaking Changes Summary

Change	v1	v2
Endpoint	`/api/imagery/infrared/query` `/api/imagery/visible/query`	`/api/imagery/q`
Image type	URL path	`?imageType=infrared\|visible`
Date params	`search_from`, `search_to`	`datefrom`, `dateto`
Date format	ISO 8601 (`2022-10-01T00:00`)	Compact (`yyyymmddhhmmss`)
Response	Array `[{...}, {...}]`	Object with pagination `{results: [...], pagination: {...}}`
Download limit	Fixed 100 images	Configurable up to 200

Migration Steps

Step 1: Update Endpoint URLs

v1: Separate endpoints

# Infrared images
https://avert-legacy.ldeo.columbia.edu/api/imagery/infrared/query

# Visible images
https://avert-legacy.ldeo.columbia.edu/api/imagery/visible/query

v2: Single unified endpoint

# Infrared images
https://avert.ldeo.columbia.edu/api/imagery/q?imageType=infrared

# Visible images
https://avert.ldeo.columbia.edu/api/imagery/q?imageType=visible

# Separate endpoints for each image type
infrared_url = "https://avert-legacy.ldeo.columbia.edu/api/imagery/infrared/query"
visible_url = "https://avert-legacy.ldeo.columbia.edu/api/imagery/visible/query"

infrared_images = requests.get(infrared_url, params=params).json()
visible_images = requests.get(visible_url, params=params).json()

# Single endpoint with imageType parameter
base_url = "https://avert.ldeo.columbia.edu/api/imagery/q"

infrared_images = requests.get(base_url, params={**params, 'imageType': 'infrared'}).json()
visible_images = requests.get(base_url, params={**params, 'imageType': 'visible'}).json()

Step 2: Update Parameter Names

Date range parameters have new names and format:

v1 Parameter	v2 Parameter	Format Change
`search_from`	`datefrom`	ISO 8601 → `yyyymmddhhmmss`
`search_to`	`dateto`	ISO 8601 → `yyyymmddhhmmss`

params = {
    'site': 'CLNE',
    'vnum': 311240,
    'search_from': '2022-10-01T00:00',
    'search_to': '2022-10-02T00:00'
}

params = {
    'site': 'CLNE',
    'vnum': 311240,
    'datefrom': '20221001000000',
    'dateto': '20221002000000'
}

Helper function for date conversion:

from datetime import datetime

def convert_to_v2_date(iso_date_str):
    """Convert ISO 8601 date to v2 format"""
    # Parse ISO 8601 format (e.g., "2022-10-01T00:00")
    dt = datetime.fromisoformat(iso_date_str)
    # Return v2 format (e.g., "20221001000000")
    return dt.strftime('%Y%m%d%H%M%S')

# Usage
v1_date = '2022-10-01T00:00'
v2_date = convert_to_v2_date(v1_date)
print(v2_date)  # '20221001000000'

function convertToV2Date(isoDateStr) {
  // Parse ISO 8601 format (e.g., "2022-10-01T00:00")
  const date = new Date(isoDateStr);
  
  // Return v2 format (e.g., "20221001000000")
  const year = date.getFullYear();
  const month = String(date.getMonth() + 1).padStart(2, '0');
  const day = String(date.getDate()).padStart(2, '0');
  const hours = String(date.getHours()).padStart(2, '0');
  const minutes = String(date.getMinutes()).padStart(2, '0');
  const seconds = String(date.getSeconds()).padStart(2, '0');
  
  return `${year}${month}${day}${hours}${minutes}${seconds}`;
}

// Usage
const v1Date = '2022-10-01T00:00';
const v2Date = convertToV2Date(v1Date);
console.log(v2Date);  // '20221001000000'

Step 3: Update Response Handling

v1: Returns array directly

[
  {
    "image_id": "...",
    "image_url": "...",
    ...
  },
  ...
]

v2: Returns structured object

{
  "results": [
    {
      "image_id": "...",
      "image_url": "...",
      ...
    },
    ...
  ],
  "pagination": {
    "page": 1,
    "limit": 100,
    "total_count": 1500,
    "has_next": true,
    ...
  },
  "query": {...}
}

response = requests.get(url, params=params)
images = response.json()  # Direct array

# Iterate images
for img in images:
    print(img['image_url'])

response = requests.get(url, params=params)
data = response.json()
images = data['results']  # Extract results array

# Iterate images
for img in images:
    print(img['image_url'])

# Access pagination info
if data['pagination']['has_next']:
    next_page = data['pagination']['next_page']
    print(f"Load page {next_page} for more results")

const response = await fetch(url);
const images = await response.json();  // Direct array

// Iterate images
images.forEach(img => {
  console.log(img.image_url);
});

const response = await fetch(url);
const data = await response.json();
const images = data.results;  // Extract results array

// Iterate images
images.forEach(img => {
  console.log(img.image_url);
});

// Access pagination info
if (data.pagination.has_next) {
  const nextPage = data.pagination.next_page;
  console.log(`Load page ${nextPage} for more results`);
}

Step 4: Update Download Logic

v1: Fixed 100-image limit

# v1: Downloads max 100 images
response = requests.get(url, params={'download': True})

v2: Estimate first, then download

# v2: Get estimate first
estimate_response = requests.get(url, params={**params, 'download': 'estimate'})
estimate = estimate_response.json()

print(f"Will download {estimate['total_images']} images ({estimate['estimated_size_mb']} MB)")

# Then download if confirmed
if user_confirms:
    download_response = requests.get(url, params={**params, 'download': 'true'})
    with open('images.zip', 'wb') as f:
        f.write(download_response.content)

def download_images_v2(params):
    """Download images with v2 API (with estimate)"""
    base_url = 'https://avert.ldeo.columbia.edu/api/imagery/q'
    
    # Step 1: Get estimate
    estimate_params = {**params, 'download': 'estimate'}
    estimate_response = requests.get(base_url, params=estimate_params)
    estimate = estimate_response.json()
    
    total = estimate['total_images']
    size_mb = estimate['estimated_size_mb']
    time_sec = estimate['estimated_time_seconds']
    
    # Step 2: Show estimate and confirm
    print(f"Download estimate:")
    print(f"  - Images: {total}")
    print(f"  - Size: {size_mb} MB")
    print(f"  - Time: {time_sec} seconds")
    
    if estimate.get('warning'):
        print(f"  - Warning: {estimate['suggestion']}")
    
    confirm = input("Proceed with download? (y/n): ")
    
    if confirm.lower() == 'y':
        # Step 3: Download
        print(f"Downloading {total} images...")
        download_params = {**params, 'download': 'true'}
        download_response = requests.get(base_url, params=download_params)
        
        with open('avert_images.zip', 'wb') as f:
            f.write(download_response.content)
        
        print(f"✓ Downloaded to avert_images.zip")
    else:
        print("Download cancelled")

# Usage
params = {
    'imageType': 'infrared',
    'site': 'CLNE',
    'datefrom': '20251101000000',
    'dateto': '20251130235959',
    'freq': 'hourly'  # NEW: Use temporal sampling
}
download_images_v2(params)

async function downloadImagesV2(params) {
  const baseUrl = 'https://avert.ldeo.columbia.edu/api/imagery/q';
  
  // Step 1: Get estimate
  const estimateParams = new URLSearchParams({
    ...params,
    download: 'estimate'
  });
  
  const estimateResponse = await fetch(`${baseUrl}?${estimateParams}`);
  const estimate = await estimateResponse.json();
  
  const { total_images, estimated_size_mb, estimated_time_seconds, warning, suggestion } = estimate;
  
  // Step 2: Show estimate and confirm
  let message = `Download ${total_images} images (${estimated_size_mb} MB)?\n`;
  message += `Estimated time: ${estimated_time_seconds} seconds`;
  
  if (warning) {
    message += `\n\n⚠️ ${suggestion}`;
  }
  
  if (confirm(message)) {
    // Step 3: Trigger download
    const downloadParams = new URLSearchParams({
      ...params,
      download: 'true'
    });
    
    // Show loading indicator
    showLoadingSpinner(`Preparing ${total_images} images...`);
    
    // Trigger browser download
    window.location.href = `${baseUrl}?${downloadParams}`;
  }
}

// Usage
const params = {
  imageType: 'infrared',
  site: 'CLNE',
  datefrom: '20251101000000',
  dateto: '20251130235959',
  freq: 'hourly'  // NEW: Use temporal sampling
};

downloadImagesV2(params);

New Features in v2

1. Condition Filters

v2 adds new filters for image quality and visibility conditions:

# Infrared filters
params = {
    'imageType': 'infrared',
    'site': 'CLNE',
    'is_degraded': False,      # NEW: Non-degraded only (infrared only)
    'low_visibility': False,   # NEW: Clear visibility (replaces has_fog/has_clouds)
    'is_empty': False,         # NEW: Non-empty files only
    'limit': 50
}

# Visible filters
params = {
    'imageType': 'visible',
    'site': 'CLNE',
    'is_night': False,         # NEW: Daytime only (visible only)
    'low_visibility': False,   # NEW: Clear visibility
    'is_empty': False,         # NEW: Non-empty files only
    'limit': 50
}

Filter applicability by image type:

Filter	Infrared	Visible	Description
`is_empty`	Yes	Yes	Empty/corrupted files
`is_night`	Error	Yes	Nighttime (IR cameras work in darkness)
`is_degraded`	Yes	Error	Dynamic range artifacts (IR only)
`low_visibility`	Yes	Yes	Fog/clouds obstruction

2. Temporal Sampling (freq)

Dramatically reduce dataset size while maintaining temporal coverage:

# Get one image per hour (60x data reduction)
params = {
    'site': 'VPMI',
    'freq': 'hourly',          # NEW: Temporal sampling
    'datefrom': '20251101000000',
    'dateto': '20251130235959'
}

# Get daily summaries (1440x data reduction)
params = {
    'site': 'VPMI',
    'freq': 'daily',           # NEW: One per day
    'datefrom': '20250101000000',
    'dateto': '20251231235959'
}

freq options:

all - All captured images (default)
minutely - One per minute
hourly - One per hour
daily - One per day

3. Enhanced Pagination

v2 provides comprehensive pagination metadata:

data = requests.get(url, params=params).json()

pagination = data['pagination']
print(f"Page {pagination['page']} of {pagination['total_pages']}")
print(f"Showing {pagination['count']} of {pagination['total_count']} results")

if pagination['has_next']:
    # Load next page
    next_page_params = {**params, 'page': pagination['next_page']}

4. Query Echo

v2 includes your query parameters in the response for debugging:

data = requests.get(url, params=params).json()

print("Your query:")
print(data['query'])
# Shows exactly what filters were applied

Complete Migration Example

Here’s a complete before/after example:

import requests

# v1 API
url = "https://avert-legacy.ldeo.columbia.edu/api/imagery/infrared/query"
params = {
    'site': 'CLNE',
    'vnum': 311240,
    'search_from': '2022-10-01T00:00',
    'search_to': '2022-10-31T23:59',
    'limit': 100
}

response = requests.get(url, params=params)
images = response.json()  # Direct array

print(f"Found {len(images)} images")
for img in images[:5]:
    print(f"  - {img['image_id']}")

# Download
download_params = {**params, 'download': True}
download_response = requests.get(url, params=download_params)
with open('images.zip', 'wb') as f:
    f.write(download_response.content)

import requests

# v2 API
url = "https://avert.ldeo.columbia.edu/api/imagery/q"
params = {
    'imageType': 'infrared',
    'site': 'CLNE',
    'vnum': 311240,
    'datefrom': '20221001000000',
    'dateto': '20221031235959',
    'is_degraded': False,      # NEW: Non-degraded only
    'low_visibility': False,   # NEW: Clear images only
    'freq': 'hourly',          # NEW: One per hour
    'limit': 100
}

response = requests.get(url, params=params)
data = response.json()  # Structured response
images = data['results']

print(f"Found {len(images)} images (page {data['pagination']['page']} of {data['pagination']['total_pages']})")
for img in images[:5]:
    print(f"  - {img['image_id']}")

# Download with estimate
estimate_response = requests.get(url, params={**params, 'download': 'estimate'})
estimate = estimate_response.json()

print(f"\nDownload estimate: {estimate['total_images']} images, {estimate['estimated_size_mb']} MB")

if input("Download? (y/n): ").lower() == 'y':
    download_response = requests.get(url, params={**params, 'download': 'true'})
    with open('images.zip', 'wb') as f:
        f.write(download_response.content)
    print("✓ Downloaded")

Migration Checklist

Use this checklist to ensure complete migration:

Update endpoint URLs

Replace /api/imagery/infrared/query with /api/imagery/q?imageType=infrared
Replace /api/imagery/visible/query with /api/imagery/q?imageType=visible
Update base URL to https://avert.ldeo.columbia.edu

Update parameters

Replace search_from with datefrom
Replace search_to with dateto
Convert date format from ISO 8601 to yyyymmddhhmmss
Add imageType parameter

Update response handling

Extract data['results'] instead of using response directly
Access pagination via data['pagination']
Update loops to iterate over data['results']

Update download logic

Add download=estimate call before downloading
Show estimate to users
Change download=True to download='true' (string)

Add new features (optional)

Add condition filters (is_empty, is_night, is_degraded, low_visibility)
Add temporal sampling (freq parameter)
Implement pagination navigation
Add rate limit handling with retry logic

Test thoroughly

Test basic queries
Test with filters
Test pagination
Test downloads with estimates
Test error handling

Common Migration Issues

TypeError: 'list' object has no attribute 'results'

Issue: Trying to access .results on v1 responseSolution: Update response handling

# Wrong (v1 style with v2 API)
images = response.json()
for img in images:  # Error!
    ...

# Correct (v2 style)
data = response.json()
images = data['results']
for img in images:
    ...

400 Error: Invalid date format

Issue: Using ISO 8601 date format with v2 APISolution: Convert to yyyymmddhhmmss format

# Wrong
'search_from': '2022-10-01T00:00'

# Correct
'datefrom': '20221001000000'

Images not filtered by type

Issue: Forgot to add imageType parameterSolution: Explicitly specify image type

# Wrong - might get wrong image type
params = {'site': 'CLNE'}

# Correct
params = {'site': 'CLNE', 'imageType': 'infrared'}

Download returns JSON instead of ZIP

Issue: Using wrong value for download parameterSolution: Use string ‘true’ not boolean True

# Wrong
params = {'download': True}  # Boolean

# Correct
params = {'download': 'true'}  # String

Testing Your Migration

Test your migrated code with these queries:

Test 1: Basic Query

# Should return structured response with results array
params = {
    'imageType': 'infrared',
    'site': 'CLNE',
    'limit': 5
}
data = requests.get('https://avert.ldeo.columbia.edu/api/imagery/q', params=params).json()

assert 'results' in data
assert 'pagination' in data
assert len(data['results']) <= 5
print("✓ Test 1 passed")

Test 2: Date Filtering

# Should accept new date format
params = {
    'imageType': 'infrared',
    'site': 'CLNE',
    'datefrom': '20251101000000',
    'dateto': '20251101235959',
    'limit': 10
}
data = requests.get('https://avert.ldeo.columbia.edu/api/imagery/q', params=params).json()

assert data['query']['datefrom'] == '20251101000000'
print("✓ Test 2 passed")

Test 3: Download Estimate

# Should return estimate without downloading
params = {
    'imageType': 'infrared',
    'site': 'CLNE',
    'limit': 10,
    'download': 'estimate'
}
estimate = requests.get('https://avert.ldeo.columbia.edu/api/imagery/q', params=params).json()

assert 'total_images' in estimate
assert 'estimated_size_mb' in estimate
print(f"✓ Test 3 passed - Would download {estimate['total_images']} images")

Getting Help

API v2 Documentation

Complete v2 API reference

Query Endpoint Reference

Detailed endpoint documentation

Contact Support

Get help from the AVERT team

v1 Documentation

Legacy API reference (if needed)

Getting Started

The Single-Board Computer

Node Software

The Data Server

API Reference v2

The AVERT Data Portal

Migration Guide: v1 to v2

Overview

Why Migrate?

Unified API

Better Filtering

Smart Downloads

Improved Pagination

Breaking Changes Summary

Migration Steps

Step 1: Update Endpoint URLs

Step 2: Update Parameter Names

Step 3: Update Response Handling

Step 4: Update Download Logic

New Features in v2

1. Condition Filters

2. Temporal Sampling (freq)

4. Query Echo

Complete Migration Example

Migration Checklist

Common Migration Issues

Testing Your Migration

Test 1: Basic Query

Test 2: Date Filtering

Test 3: Download Estimate

Getting Help

API v2 Documentation

Query Endpoint Reference

Contact Support

v1 Documentation

​Overview

​Why Migrate?

Unified API

Better Filtering

Smart Downloads

Improved Pagination

​Breaking Changes Summary

​Migration Steps

​Step 1: Update Endpoint URLs

​Step 2: Update Parameter Names

​Step 3: Update Response Handling

​Step 4: Update Download Logic

​New Features in v2

​1. Condition Filters

​2. Temporal Sampling (freq)

​3. Enhanced Pagination

​4. Query Echo

​Complete Migration Example

​Migration Checklist

​Common Migration Issues

​Testing Your Migration

​Test 1: Basic Query

​Test 2: Date Filtering

​Test 3: Download Estimate

​Getting Help

API v2 Documentation

Query Endpoint Reference

Contact Support

v1 Documentation

Overview

Why Migrate?

Breaking Changes Summary

Migration Steps

Step 1: Update Endpoint URLs

Step 2: Update Parameter Names

Step 3: Update Response Handling

Step 4: Update Download Logic

New Features in v2

1. Condition Filters

2. Temporal Sampling (freq)

3. Enhanced Pagination

4. Query Echo

Complete Migration Example

Migration Checklist

Common Migration Issues

Testing Your Migration

Test 1: Basic Query

Test 2: Date Filtering

Test 3: Download Estimate

Getting Help