Architecture: Packages, Backends & Data Flow

Architecture

This document describes Stowry’s technical architecture and design decisions.

Overview

Stowry follows a clean architecture with clear separation between:

• Core domain logic (internal/service)
• Configuration (internal/config)
• Infrastructure adapters (internal/database, internal/keybackend, internal/handler, internal/middleware)
• Client (internal/client)
• Signing, shared by server and client (sign)
• Wire types, shared by server and client (types)
• CLI, one binary for both halves (internal/cli, main.go)

flowchart TB
    subgraph Clients["🖥️ Clients"]
        cli["stowry client commands"]
        sdk["SDK / HTTP"]
    end

    subgraph Server["⚙️ Server"]
        direction TB
        httpLayer["HTTP Layer"]
        auth["Auth Middleware"]
        service["Service"]
        httpLayer --> auth --> service
    end

    subgraph Storage["💾 Storage"]
        direction LR
        metadata[("Metadata")]
        files[("Files")]
    end

    cli & sdk --> httpLayer
    service --> metadata & files

    style Clients fill:#e0f2fe,stroke:#0284c7,color:#0c4a6e
    style Server fill:#fef3c7,stroke:#d97706,color:#78350f
    style Storage fill:#dcfce7,stroke:#16a34a,color:#14532d

Core Package

internal/service contains:

Interfaces

// MetaDataRepo persists object metadata
type MetaDataRepo interface {
    Get(ctx context.Context, path string) (MetaData, error)
    Upsert(ctx context.Context, entry MetaData) error
    Delete(ctx context.Context, path string) error
    List(ctx context.Context, query ListQuery) (ListResult, error)
    ListPendingCleanup(ctx context.Context, query ListQuery) (ListResult, error)
    MarkCleanedUp(ctx context.Context, id uuid.UUID) error
}

// Storage is spf13/afero's Fs, scoped to the storage directory with
// afero.NewBasePathFs, so a path cannot escape it.

Service

The main service orchestrates metadata and storage operations:

type Service struct {
    repo    MetaDataRepo
    storage afero.Fs
}

func (s *Service) Create(ctx context.Context, obj CreateObject, content io.Reader) (MetaData, error)
func (s *Service) Get(ctx context.Context, path string) (MetaData, io.ReadSeekCloser, error)
func (s *Service) Info(ctx context.Context, path string) (MetaData, error)
func (s *Service) Delete(ctx context.Context, path string) error
func (s *Service) List(ctx context.Context, query ListQuery) (ListResult, error)
func (s *Service) Populate(ctx context.Context) ([]types.MetaData, error)

Domain Types

type MetaData struct {
    ID            uuid.UUID
    Path          string
    ContentType   string
    Etag          string      // SHA256 hash
    FileSizeBytes int64
    CreatedAt     time.Time
    UpdatedAt     time.Time
}

type ServerMode string
const (
    ModeStore  ServerMode = "store"
    ModeStatic ServerMode = "static"
    ModeSPA    ServerMode = "spa"
)

Database Backends

The database package provides a unified interface for connecting to metadata backends.

Unified Connection

import (
    "github.com/sagarc03/stowry/internal/database"
    "github.com/sagarc03/stowry/types"
)

cfg := database.Config{
    Type:   "sqlite",  // or "postgres"
    DSN:    "stowry.db",
    Tables: types.Tables{MetaData: "stowry_metadata"},
}

db, err := database.Connect(ctx, cfg)
if err != nil {
    log.Fatal(err)
}
defer db.Close()

// Run migrations or validate existing schema
if err := db.Migrate(ctx); err != nil {
    log.Fatal(err)
}

SQLite (internal/database)

• Uses modernc.org/sqlite (pure Go implementation)
• Single-file database, no external process
• Code-based migrations
• Ideal for development and small deployments

PostgreSQL (internal/database)

• Uses pgx/v5 with connection pooling
• Code-based migrations
• Suitable for production and high concurrency

Schema

Both backends use equivalent schemas with database-specific types. See Configuration - Database for the full SQL if you prefer manual migrations.

PostgreSQL uses UUID, TIMESTAMPTZ, and BIGINT types with partial indexes for performance.

SQLite uses TEXT for all types (UUID as string, timestamps as ISO8601 strings).

Migrations

Migrations are code-based, not SQL files. The database provides explicit methods for migration and validation:

db, err := database.Connect(ctx, cfg)
if err != nil {
    log.Fatal(err)
}
defer db.Close()

// Run migrations (creates tables if they don't exist)
if err := db.Migrate(ctx); err != nil {
    log.Fatal(err)
}

// Validate schema matches expected structure
if err := db.Validate(ctx); err != nil {
    log.Fatal(err)
}

stowry migrate runs the migration; stowry populate records metadata for files already in the storage directory. migrate validates its own result, so a schema stowry did not build is reported rather than changed.

File Storage (internal/service, over spf13/afero)

Atomic Writes

Files are written atomically to prevent corruption:

1. Write content to temporary file
2. Calculate SHA256 hash
3. Rename temp file to final path

func (s *FileStorage) Write(ctx context.Context, path string, content io.Reader) (FileInfo, error) {
    // 1. Create temp file
    temp, err := os.CreateTemp(dir, ".stowry-*")

    // 2. Write content, calculate hash
    hasher := sha256.New()
    writer := io.MultiWriter(temp, hasher)
    io.Copy(writer, content)

    // 3. Atomic rename
    os.Rename(temp.Name(), finalPath)
}

Path Sandboxing

A base path filesystem confines every operation to the storage directory:

storage := afero.NewBasePathFs(afero.NewOsFs(), storagePath)

types.IsValidPath rejects a path before it reaches storage: anything absolute, containing .., //, a . segment, backslashes, control characters or whitespace.

Content Type Detection

MIME types are detected from file extensions using Go’s mime package.

Key Backend (internal/keybackend)

The keybackend package provides pluggable secret key storage for signature verification.

SecretStore Interface

// SecretStore provides access key lookup for signature verification.
type SecretStore interface {
    Lookup(accessKey string) (secretKey string, err error)
}

MapSecretStore

In-memory implementation suitable for configuration file-based key storage:

store := keybackend.NewMapSecretStore(map[string]string{
    "AKIAIOSFODNN7EXAMPLE": "wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY",
})

cfg := sign.AuthConfig{AWS: sign.AWSConfig{Region: "us-east-1", Service: "s3"}}
verifier := sign.NewSignatureVerifier(cfg, store)

HTTP Layer (internal/handler, internal/middleware)

Router

net/http.ServeMux, with the write routes registered only in store mode:

if opts.Mode == types.ModeStore {
    mux.Handle("GET /{$}", readAuth(HandleList(...)))
    mux.Handle("PUT /", writeAuth(HandlePut(...)))
    mux.Handle("DELETE /", writeAuth(HandleDelete(...)))
}

mux.Handle("GET /", readAuth(byMethod(HandleGet(...), HandleHead(...))))

byMethod splits GET from HEAD by hand. ServeMux cannot: a GET pattern matches HEAD too, so HEAD / would overlap GET /{$} with neither more specific, which it rejects as a conflict.

The middleware wrapping the mux, rather than each route, is WithRequestID -> WithLogging -> WithCORS, so a preflight and a 404 are covered as well. CORS sits ahead of authentication because browsers omit credentials from a preflight.

Authentication Middleware

Supports two signing schemes:

1. Stowry Native - X-Stowry-* query parameters
2. AWS Signature V4 - X-Amz-* query parameters

The middleware auto-detects which scheme to use based on query parameters.

Deletion

DELETE /file.txt

Service.Delete soft-deletes the metadata, setting deleted_at so the object drops out of listings, and then removes the stored file.

The schema also carries cleaned_up_at, and the repository exposes ListPendingCleanup and MarkCleanedUp for a two-phase delete where the file outlives the metadata entry. Nothing uses them yet: the file goes with the first phase, so no entry is ever left pending. The columns are there for a cleanup command that has not been built.

Error Handling

Sentinel Errors

internal/service/errors.go

var (
    ErrNotFound     = errors.New("not found")
    ErrInvalidInput = errors.New("invalid input")
)

// sign/errors.go
var (
    ErrMissingParams     = errors.New("missing required signature parameters")
    ErrExpired           = errors.New("signature expired")
    ErrInvalidCredential = errors.New("invalid credential")
    ErrInvalidSignature  = errors.New("invalid signature")
)

// internal/keybackend/keybackend.go
var ErrKeyNotFound = errors.New("access key not found")

// internal/client - matched with errors.Is against an *APIError
var (
    ErrNotFound     = &APIError{StatusCode: http.StatusNotFound}
    ErrUnauthorized = &APIError{StatusCode: http.StatusUnauthorized}
    ErrForbidden    = &APIError{StatusCode: http.StatusForbidden}
)

Error Wrapping

Errors are wrapped with context for debugging:

return fmt.Errorf("create object %s: %w", path, err)

HTTP Error Responses

{
  "error": "not_found",
  "message": "Object not found"
}

Pagination

Uses cursor-based pagination for consistent results:

Cursor Format

Base64(createdAt|path)

Implementation

type ListQuery struct {
    PathPrefix string
    Limit      int
    Cursor     string  // Encoded cursor
}

type ListResult struct {
    Items      []MetaData
    NextCursor string  // For next page
}

Benefits

• Handles concurrent modifications gracefully
• No offset skipping issues
• Efficient for large datasets

Configuration

Precedence

1. Default values
2. Config file (config.yaml)
3. Environment variables (STOWRY_*)
4. Command-line flags

Architecture Separation

Configuration logic is handled by the config package, which the CLI uses:

// internal/config - loading and validation
cfg, err := config.Load(configFiles, cmd.Flags())

// Config is passed through context
cmd.SetContext(config.WithContext(ctx, cfg))

// internal/service - no config awareness
svc := service.New(db, storage)

Testing

Unit Tests

• Mock interfaces with testify/mock
• Black-box testing from an external _test package
• Table-driven tests

Integration Tests

• PostgreSQL: testcontainers-go for real database
• SQLite: In-memory database (:memory:)

Example Test Structure

func TestStowryService_Create(t *testing.T) {
    tests := []struct {
        name    string
        setup   func(*SpyMetaDataRepo, *SpyFileStorage)
        obj     CreateObject
        content string
        wantErr error
    }{
        // Test cases...
    }

    for _, tt := range tests {
        t.Run(tt.name, func(t *testing.T) {
            // Setup, execute, assert
        })
    }
}

Design Decisions

Why Not S3 API Compatible?

Stowry intentionally doesn’t implement the full S3 API:

1. Simplicity - Smaller codebase, easier to maintain
2. Focused - Only presigned URL authentication
3. Flexibility - Can evolve independently

Why Presigned URLs?

1. Security - Credentials never exposed to clients
2. Scalability - No session state on server
3. Compatibility - Works with AWS SDK presigning