Storage

S3-compatible file storage with buckets and presigned URLs

Overview

InsForge provides S3-compatible object storage for managing files. Files are organized in buckets with public or private access control.

Key Features

• S3-Compatible - Works with any S3-compatible storage backend
• Buckets - Organize files into logical containers
• Public/Private Access - Control file visibility
• Presigned URLs - Secure temporary access to private files
• Direct Upload - Upload large files directly to S3
• Auto-generated Keys - Unique filenames to prevent collisions

Architecture

Bucket Management

Creating Buckets

Buckets must be created before uploading files.

MCP Tool (Recommended)

// Use the InsForge MCP create-bucket tool
// This is the recommended way for infrastructure setup

Response:

{
  "message": "Bucket created successfully",
  "bucketName": "avatars"
}

Bucket Visibility

Type	Access	Use Case
Public	Direct URLs, no auth	Profile pictures, public assets
Private	Presigned URLs with expiration	User documents, sensitive files

List Buckets

curl https://your-app.region.insforge.app/api/storage/buckets \
  -H "Authorization: Bearer YOUR_ADMIN_TOKEN"

Response:

{
  "buckets": [
    "avatars",
    "documents",
    "uploads"
  ]
}

Update Bucket Visibility

curl -X PATCH https://your-app.region.insforge.app/api/storage/buckets/avatars \
  -H "Authorization: Bearer YOUR_ADMIN_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"isPublic": false}'

Delete Bucket

Deletes all files in the bucket permanently.

curl -X DELETE https://your-app.region.insforge.app/api/storage/buckets/avatars \
  -H "Authorization: Bearer YOUR_ADMIN_TOKEN"

Uploading Files

InsForge provides two upload strategies:

1. Direct Upload - For local storage, upload directly to backend
2. Presigned URL - For S3, upload directly to storage provider

Simple Upload (SDK)

TypeScript SDK

import { createClient } from '@insforge/sdk';

const client = createClient({
  baseUrl: 'https://your-app.region.insforge.app',
  anonKey: 'your-anon-key'
});

// Upload file to bucket
const { data, error } = await client.storage
  .from('avatars')
  .upload(file);

if (error) {
  console.error('Upload failed:', error);
} else {
  console.log('File uploaded:', data);
  console.log('URL:', data.url);
}

Advanced Upload (Presigned URLs)

For large files, get upload strategy first:

1. Get Upload Strategy

   const { data: strategy } = await client.storage
     .from('avatars')
     .getUploadStrategy({
       filename: file.name,
       contentType: file.type,
       size: file.size
     });
   
   console.log('Method:', strategy.method); // 'presigned' or 'direct'
   console.log('Upload URL:', strategy.uploadUrl);
   
2. Upload to Storage

   For S3 presigned POST:

   if (strategy.method === 'presigned') {
     const formData = new FormData();
     
     // Add presigned fields
     Object.entries(strategy.fields).forEach(([key, value]) => {
       formData.append(key, value);
     });
     
     // Add file last
     formData.append('file', file);
     
     // Upload to S3
     await fetch(strategy.uploadUrl, {
       method: 'POST',
       body: formData
     });
   }
   
3. Confirm Upload

   if (strategy.confirmRequired) {
     const { data, error } = await fetch(strategy.confirmUrl, {
       method: 'POST',
       headers: {
         'Content-Type': 'application/json',
         'Authorization': `Bearer ${accessToken}`
       },
       body: JSON.stringify({
         size: file.size,
         contentType: file.type
       })
     });
   }
   

Downloading Files

Public Files

Public files can be accessed directly:

const publicUrl = `${baseUrl}/api/storage/buckets/avatars/objects/user-123.jpg`;

// Use in img tag
<img src={publicUrl} alt="Avatar" />

Private Files

Get temporary signed URL for private files:

TypeScript SDK

const { data, error } = await client.storage
  .from('documents')
  .download('invoice-2024.pdf');

if (!error) {
  console.log('Download URL:', data.url);
  console.log('Expires at:', data.expiresAt);
}

Download Strategy Response

interface DownloadStrategy {
  method: 'direct' | 'presigned';
  url: string;
  expiresAt?: string; // For presigned URLs
  headers?: Record<string, string>;
}

Listing Files

List All Files in Bucket

TypeScript SDK

const { data, error } = await client.storage
  .from('avatars')
  .list();

if (!error) {
  data.forEach(file => {
    console.log('File:', file.key);
    console.log('Size:', file.size);
    console.log('URL:', file.url);
  });
}

Response:

{
  "data": [
    {
      "bucket": "avatars",
      "key": "user-123.jpg",
      "size": 102400,
      "mimeType": "image/jpeg",
      "uploadedAt": "2024-01-15T10:30:00Z",
      "url": "/api/storage/buckets/avatars/objects/user-123.jpg"
    }
  ],
  "pagination": {
    "offset": 0,
    "limit": 100,
    "total": 1
  }
}

Filtering and Pagination

Prefix Filter

// List files in specific folder
const { data } = await client.storage
  .from('documents')
  .list({
    prefix: 'invoices/'
  });

Deleting Files

TypeScript SDK

const { error } = await client.storage
  .from('avatars')
  .remove('user-123.jpg');

if (!error) {
  console.log('File deleted');
}

File Organization

Using Pseudo-folders

Organize files with forward slashes in keys:

// Upload to pseudo-folder
await client.storage
  .from('documents')
  .upload(file, {
    key: 'users/123/invoices/2024-01.pdf'
  });

// List files in folder
const { data } = await client.storage
  .from('documents')
  .list({ prefix: 'users/123/invoices/' });

Naming Conventions

// Auto-generated key format:
// {filename}-{timestamp}-{random}.{ext}
// Example: profile-1737546841234-a3f2b1.jpg

// Custom key patterns:
const key = `users/${userId}/avatar-${Date.now()}.jpg`;
const key = `documents/${year}/${month}/${filename}`;

Integration with Database

Store file URLs in database after upload:

Upload & Save URL

// 1. Upload file
const { data: file, error: uploadError } = await client.storage
  .from('avatars')
  .upload(avatarFile);

if (uploadError) throw uploadError;

// 2. Save URL to database
const { error: dbError } = await client.database
  .from('profiles')
  .update({
    avatar_url: file.url
  })
  .eq('user_id', userId);

if (dbError) throw dbError;

File Types

Common MIME types:

Type	MIME Type	Extension
JPEG	image/jpeg	.jpg, .jpeg
PNG	image/png	.png
GIF	image/gif	.gif
WebP	image/webp	.webp
PDF	application/pdf	.pdf
JSON	application/json	.json
CSV	text/csv	.csv
ZIP	application/zip	.zip

Storage Quotas

Contact InsForge support for storage quota limits and pricing.

API Reference

Endpoints

# Bucket management (Admin)
GET    /api/storage/buckets
POST   /api/storage/buckets
DELETE /api/storage/buckets/{bucketName}
PATCH  /api/storage/buckets/{bucketName}

# File operations
GET    /api/storage/buckets/{bucket}/objects
POST   /api/storage/buckets/{bucket}/objects
PUT    /api/storage/buckets/{bucket}/objects/{key}
GET    /api/storage/buckets/{bucket}/objects/{key}
DELETE /api/storage/buckets/{bucket}/objects/{key}

# Upload/Download strategies
POST   /api/storage/buckets/{bucket}/upload-strategy
POST   /api/storage/buckets/{bucket}/objects/{key}/download-strategy
POST   /api/storage/buckets/{bucket}/objects/{key}/confirm-upload

File Metadata

interface StoredFile {
  bucket: string;
  key: string;
  size: number;
  mimeType: string;
  uploadedAt: string;
  url: string;
}

Best Practices

Use Public Buckets for Assets

Store profile pictures and public assets in public buckets for faster access

Prefix Keys with User ID

Organize user files: users/{userId}/documents/file.pdf

Store URLs in Database

Save file URLs in your database for easy querying and relationships

Set Appropriate Expiration

Use short expiration times (1 hour) for sensitive presigned URLs

Validate File Types

Check MIME types on client and server to prevent malicious uploads

Next Steps

Database

Store file metadata and URLs

Authentication

Secure file uploads with auth

Functions

Process uploaded files

AI Integration

Generate images with AI