diff --git a/docs/IOR_RESOLVER.md b/docs/IOR_RESOLVER.md new file mode 100644 index 0000000..59cf72d --- /dev/null +++ b/docs/IOR_RESOLVER.md @@ -0,0 +1,617 @@ +# IOR Core Resolver - Implementation Guide + +## Overview + +The IOR Core Resolver provides a unified architecture for resolving Interoperable Object References (IOR) across different JavaScript environments. This document serves as a technical guide for implementing IOR resolution in bundlers and loaders beyond the existing Metro and Node.js implementations. + +## Core Architecture + +The IOR resolver architecture consists of three main layers: + +### 1. Shared Core (`ior-core.js`) +The foundation that provides: +- IOR string parsing and validation +- Remote component fetching (GitHub, Gitea, IPFS) +- Local component discovery +- Caching mechanisms +- Archive extraction and entry point resolution + +### 2. Environment-Specific Implementations +- **Metro Resolver** (`metro-ior-resolver.js`) - React Native bundler integration +- **Node Import Loader** (`NodeImportLoader.ts`) - Native Node.js module loading +- **Future Implementations** - Webpack, Vite, Rollup, etc. + +### 3. Supporting Tools +- **Pre-build Type Generator** (`prebuild-ior-types.js`) - TypeScript declaration generation +- **Remote Component Fetcher** (`fetch-remote-component.js`) - Standalone fetching utility + +## Implementation Requirements + +When implementing IOR resolution for a new bundler or loader, ensure the following requirements are met: + +### 1. Module Resolution Hook +Your bundler must provide a way to intercept module resolution requests: +```javascript +// Example pattern +function resolveRequest(context, moduleName, platform) { + if (moduleName.startsWith('ior:')) { + return resolveIORModule(moduleName); + } + return defaultResolve(moduleName); +} +``` + +### 2. Synchronous or Asynchronous Support +Depending on your bundler's architecture: +- **Synchronous** (like Metro): Use `sync-fetch` or pre-cached components +- **Asynchronous** (like Node.js loader): Use native `fetch` or `https` modules + +### 3. File System Access +Required for: +- Scanning local components +- Managing cache directories +- Extracting downloaded archives + +## Core Functions Reference + +The `ior-core.js` module exposes these essential functions for implementing IOR resolution: + +### Parsing & Validation +```javascript +parseRemoteIOR(ior) // Parse remote IOR strings +parsePackageJson(path) // Sync parse package.json +parsePackageJsonAsync(path) // Async parse package.json +``` + +### Remote Fetching +```javascript +fetchUrl(url) // Fetch text content +fetchUrlBuffer(url) // Fetch as Buffer +fetchFromGitHub(config) // GitHub-specific fetching +fetchFromGitea(config) // Gitea-specific fetching +fetchFromIPFS(config) // IPFS gateway fetching +``` + +### Local Resolution +```javascript +buildLocalIORMappings(root) // Scan and map local components +findEntryPoint(dir, name) // Locate component entry file +``` + +### Caching & Extraction +```javascript +generateCacheHash(ior) // SHA256 hash for cache keys +extractArchive(buffer, dir) // Extract tar.gz archives +``` + +## Implementation Examples + +### Metro Resolver Pattern +```javascript +const { buildLocalIORMappings, fetchRemoteComponent } = require('@metatrom/ior-resolver/core'); + +function createResolver(projectRoot) { + const mappings = buildLocalIORMappings(projectRoot); + + return (context, moduleName, platform) => { + if (moduleName.startsWith('ior:')) { + // Check local mappings first + if (mappings[moduleName]) { + return { filePath: mappings[moduleName] }; + } + + // Fetch remote component synchronously + const cachedPath = fetchRemoteComponentSync(moduleName); + return { filePath: cachedPath }; + } + // Default resolution + return context.resolveRequest(context, moduleName, platform); + }; +} +``` + +### Node.js Loader Pattern +```javascript +import { parseRemoteIOR, fetchFromGitHub } from '@metatrom/ior-resolver/core'; + +export async function resolve(specifier, context, defaultResolve) { + if (specifier.startsWith('ior:')) { + const config = parseRemoteIOR(specifier); + if (config) { + const localPath = await fetchAndCache(config); + return { url: pathToFileURL(localPath).href }; + } + } + return defaultResolve(specifier, context, defaultResolve); +} +``` + +## IOR Format Specification + +### Format Structure +The IOR format follows a consistent pattern across all sources: +``` +ior:[protocol]:[location]:[component]@[version] +``` + +### Protocol Types + +| Protocol | Description | Location Format | Example | +|----------|-------------|-----------------|---------|| +| `esm` | Local ESM modules | Component namespace | `ior:esm:com.example.logger@1.0.0` | +| `github` | GitHub repositories | `owner/repo` | `ior:github:facebook/react:scheduler@0.23.0` | +| `gitea` | Gitea instances | `instance:owner/repo` | `ior:gitea:git.company.com:team/auth@2.0.0` | +| `ipfs` | IPFS network | Content hash | `ior:ipfs:QmHash:component@1.0.0` | +| `p2p` | P2P network (planned) | Peer ID | `ior:p2p:peer-id:component@1.0.0` | + +### Version Resolution +- **Git-based** (GitHub/Gitea): Uses Git tags (`1.0.0` or `v1.0.0`) +- **Local**: Matches `package.json` version field +- **IPFS**: Version included in path structure +- **P2P**: To be determined in future implementation + +## Resolution Algorithm + +Implement IOR resolution following this algorithm: + +```javascript +function resolveIOR(ior, options = {}) { + // Step 1: Validate IOR format + if (!ior.startsWith('ior:')) { + return null; + } + + // Step 2: Check if local component + if (ior.includes(':esm:')) { + return resolveLocalComponent(ior, options); + } + + // Step 3: Parse remote IOR + const config = parseRemoteIOR(ior); + if (!config) { + throw new Error(`Invalid IOR format: ${ior}`); + } + + // Step 4: Check cache + const cachedPath = checkCache(config, options.cacheTTL); + if (cachedPath) { + return cachedPath; + } + + // Step 5: Fetch and cache + const fetchedPath = fetchAndCache(config, options); + return fetchedPath; +} +``` + +### Resolution Steps + +1. **Format Validation**: Ensure IOR follows expected pattern +2. **Local vs Remote**: Determine resolution strategy +3. **Cache Check**: Look for valid cached version +4. **Remote Fetch**: Download if not cached +5. **Entry Point**: Locate main file in component +6. **Path Return**: Provide absolute path to bundler + +## Caching Implementation + +### Cache Strategy +Implement a two-tier caching system: + +1. **Memory Cache** (Optional) + - Short-lived resolution cache + - Avoids repeated file system checks + - Clear on file changes + +2. **File System Cache** (Required) + - Persistent component storage + - TTL-based invalidation + - Archive extraction location + +### Cache Structure +``` +.ior-cache/ +├── remote/ +│ ├── {sha256-hash}/ # Extracted component +│ │ ├── package.json +│ │ ├── index.ts +│ │ └── component.d.ts +│ └── metadata.json # Cache metadata +└── mappings.json # IOR to path mappings +``` + +### Cache Metadata +```json +{ + "ior": "ior:github:owner/repo:component@1.0.0", + "cachedAt": "2025-08-28T10:00:00Z", + "ttl": 3600000, + "entryPoint": "index.ts", + "hash": "sha256-hash" +} +``` + +### Cache Location Considerations + +| Environment | Recommended Location | Reason | +|-------------|---------------------|---------|| +| Metro | Project `.ior-cache/` | Watchman compatibility | +| Node.js | OS temp directory | System conventions | +| Webpack | `node_modules/.cache/` | Standard practice | +| Vite | `node_modules/.vite/` | Framework convention | + +## Bundler Integration Examples + +### Webpack Plugin (Conceptual) +```javascript +class IORResolverPlugin { + constructor(options = {}) { + this.cacheDir = options.cacheDir || '.ior-cache'; + this.ttl = options.ttl || 3600000; + } + + apply(compiler) { + compiler.hooks.resolve.tapAsync('IORResolver', (request, resolveContext, callback) => { + if (request.request?.startsWith('ior:')) { + const resolved = this.resolveIOR(request.request); + if (resolved) { + request.path = resolved; + return callback(); + } + } + callback(); + }); + } +} +``` + +### Vite Plugin (Conceptual) +```javascript +export function iorResolver(options = {}) { + return { + name: 'vite-plugin-ior', + async resolveId(source) { + if (source.startsWith('ior:')) { + const resolved = await resolveIOR(source, options); + return resolved; + } + return null; + } + }; +} +``` + +## Testing Your Implementation + +### Unit Tests +```javascript +// Test IOR parsing +test('parseRemoteIOR', () => { + const config = parseRemoteIOR('ior:github:owner/repo:component@1.0.0'); + expect(config).toEqual({ + protocol: 'github', + owner: 'owner', + repo: 'repo', + component: 'component', + version: '1.0.0' + }); +}); + +// Test cache functionality +test('caching', async () => { + const ior = 'ior:github:test/repo:component@1.0.0'; + const path1 = await resolveIOR(ior); + const path2 = await resolveIOR(ior); + expect(path1).toBe(path2); // Should use cache +}); +``` + +### Integration Tests +```javascript +// Test with actual bundler +test('bundler integration', async () => { + const bundle = await bundler.build({ + entry: './test-ior-imports.js', + plugins: [iorResolverPlugin()] + }); + expect(bundle.modules).toContain('ior:github:owner/repo:component@1.0.0'); +}); +``` + +## Performance Optimization + +### Bundler-Specific Optimizations + +1. **Pre-fetching Strategy** + - Scan codebase for IOR imports before bundling + - Fetch all remote components in parallel + - Generate type declarations upfront + +2. **Async vs Sync Resolution** + - Use async when bundler supports it (Webpack, Vite) + - Implement sync with pre-cached components (Metro) + - Consider worker threads for CPU-intensive operations + +3. **Cache Warming** + ```javascript + // Pre-warm cache during CI/CD + const iors = scanForIORImports('./src'); + await Promise.all(iors.map(fetchRemoteComponent)); + ``` + +4. **Memory Optimization** + - Implement LRU cache for memory mappings + - Stream large archives during extraction + - Clean expired cache entries periodically + +## Type Generation Strategy + +### Automatic Type Generation +Implement type generation following this approach: + +1. **Scan Phase** + ```javascript + const iorImports = scanSourceFiles({ + root: './src', + extensions: ['.ts', '.tsx', '.js', '.jsx'], + pattern: /from\s+['"]ior:[^'"]+['"]/g + }); + ``` + +2. **Fetch & Extract Types** + ```javascript + for (const ior of iorImports) { + const cachePath = await fetchComponent(ior); + const types = extractTypeDefinitions(cachePath); + typeMap[ior] = types; + } + ``` + +3. **Generate Declaration File** + ```typescript + // auto-generated.d.ts + declare module 'ior:github:owner/repo:component@1.0.0' { + export interface ComponentProps { /* extracted */ } + export const Component: React.FC; + } + ``` + +## Implementation Checklist + +When implementing IOR resolution for a new bundler: + +### Required Features +- [ ] IOR format parsing +- [ ] Local component resolution +- [ ] Remote component fetching (at least one protocol) +- [ ] File system caching with TTL +- [ ] Entry point resolution +- [ ] Error handling and logging + +### Recommended Features +- [ ] TypeScript declaration generation +- [ ] Pre-build scanning and fetching +- [ ] Hot reload support for local components +- [ ] Cache invalidation mechanisms +- [ ] Parallel fetching for multiple components +- [ ] Progress reporting for large downloads + +### Optional Enhancements +- [ ] Memory caching layer +- [ ] Custom protocol support +- [ ] Private repository authentication +- [ ] Cache compression +- [ ] Dependency graph analysis +- [ ] Build-time optimization + +## Component Publishing Guidelines + +### Repository Structure +Components should follow this structure for optimal compatibility: + +``` +component-repo/ +├── package.json # Component metadata +├── index.ts # Main entry (or specified in package.json) +├── index.d.ts # TypeScript declarations +├── README.md # Documentation +└── src/ # Additional source files (optional) + └── ... +``` + +### Metadata Requirements + +#### Minimal package.json +```json +{ + "name": "component-name", + "version": "1.0.0", + "main": "index.ts", + "types": "index.d.ts" +} +``` + +#### Full Metatrom Integration +```json +{ + "name": "@org/component", + "version": "1.0.0", + "main": "index.ts", + "types": "index.d.ts", + "type": "module", + "metatrom": { + "ior": "com.example.component@1.0.0", + "protocols": ["github", "gitea", "ipfs"], + "capabilities": { + "platform": ["react-native", "web", "node"], + "features": ["typescript", "async"] + } + } +} +``` + +### Versioning Best Practices +1. Use semantic versioning (MAJOR.MINOR.PATCH) +2. Tag releases in Git: `git tag v1.0.0` or `git tag 1.0.0` +3. Include CHANGELOG.md for version history +4. Ensure backward compatibility within minor versions + +## Error Handling Guidelines + +### Error Categories + +1. **Resolution Errors** + ```javascript + class IORResolutionError extends Error { + constructor(ior, reason) { + super(`Failed to resolve ${ior}: ${reason}`); + this.name = 'IORResolutionError'; + this.ior = ior; + } + } + ``` + +2. **Network Errors** + - Retry with exponential backoff + - Fall back to alternative gateways (IPFS) + - Provide clear error messages with recovery steps + +3. **Cache Errors** + - Auto-clear corrupted cache entries + - Fall back to re-fetching + - Log cache operations for debugging + +### Error Recovery Strategies + +```javascript +async function resolveWithRetry(ior, maxRetries = 3) { + let lastError; + + for (let i = 0; i < maxRetries; i++) { + try { + return await resolveIOR(ior); + } catch (error) { + lastError = error; + + // Clear cache if corrupted + if (error.code === 'CACHE_CORRUPTED') { + clearCache(ior); + } + + // Wait before retry + await delay(Math.pow(2, i) * 1000); + } + } + + throw new IORResolutionError(ior, lastError.message); +} +``` + +## Security Considerations + +### Component Validation + +1. **Source Verification** + - Validate HTTPS certificates for Git sources + - Verify IPFS content hashes + - Check component signatures (future) + +2. **Content Sanitization** + - Scan for malicious patterns + - Validate package.json structure + - Restrict file system access + +3. **Sandboxing Options** + ```javascript + const secureResolver = createIORResolver({ + allowedProtocols: ['github', 'gitea'], + allowedDomains: ['github.com', 'git.company.com'], + maxComponentSize: 10 * 1024 * 1024, // 10MB + scanForMalware: true + }); + ``` + +### Authentication Support + +For private repositories: + +```javascript +const resolver = createIORResolver({ + auth: { + github: process.env.GITHUB_TOKEN, + gitea: { + 'git.company.com': process.env.GITEA_TOKEN + } + } +}); +``` + +## Platform-Specific Considerations + +### React Native (Metro) +- Synchronous resolution required +- Watchman integration for file watching +- SHA-1 hashing for module identification +- Bundle-time resolution only + +### Node.js +- Async resolution preferred +- Native ES modules support +- TypeScript transpilation needed +- Runtime resolution possible + +### Web Bundlers (Webpack/Vite) +- Plugin architecture varies +- Build-time optimization opportunities +- Tree-shaking considerations +- Source map generation + +### Cross-Platform Compatibility + +Ensure your implementation handles: +- Path separators (`/` vs `\`) +- Case sensitivity differences +- Symlink support variations +- File system permissions + +## Future Roadmap + +### Short Term (v2.0) +- [ ] Private repository authentication +- [ ] Improved error messages and recovery +- [ ] Performance metrics and monitoring +- [ ] Plugin templates for popular bundlers + +### Medium Term (v3.0) +- [ ] P2P protocol implementation (libp2p) +- [ ] Component dependency resolution +- [ ] Versioning conflict resolution +- [ ] Cross-platform component compatibility checks + +### Long Term +- [ ] Decentralized component registry +- [ ] Smart contract integration for Web3 +- [ ] Component marketplace +- [ ] AI-assisted component discovery + +## Contributing + +To add support for a new bundler: + +1. Study the `ior-core.js` implementation +2. Create bundler-specific adapter using core functions +3. Implement required resolver hooks +4. Add comprehensive tests +5. Document integration steps +6. Submit PR with examples + +## Resources + +- [IOR Specification](https://github.com/metatrom/ior-spec) +- [Core Implementation](https://github.com/metatrom/ior-resolver) +- [Example Components](https://github.com/metatrom/universal-components) +- [Integration Examples](https://github.com/metatrom/ior-examples) + +## Conclusion + +The IOR Core Resolver architecture provides a robust foundation for implementing component resolution across any JavaScript bundler or loader. By following this guide and leveraging the shared core functions, you can enable IOR support in your build system while maintaining consistency with existing implementations. \ No newline at end of file