| 1 | diff --git a/docs/files-and-private-repositories.md b/docs/files-and-private-repositories.md |
| 2 | index 1259d4c..89292d6 100644 |
| 3 | --- a/docs/files-and-private-repositories.md |
| 4 | +++ b/docs/files-and-private-repositories.md |
| 5 | @@ -1,34 +1,36 @@ |
| 6 | # Files and private repositories |
| 7 | |
| 8 | -Use **Files** to browse uploads by name, open folders, preview images and download files. **My files** shows your uploads. **Sites** groups published assets by site, and **Rooms** groups attachments by rooms you can access. Owners and moderators can open **Storage** for the full inventory. |
| 9 | +Use **Files** to browse uploads by name, open folders, preview images and play supported audio and video files. **My files** shows your uploads and files shared with relay members. **Sites** groups published assets by site, and **Rooms** groups attachments by rooms you can access. Owners and moderators can open **Storage** for the full inventory. |
| 10 | |
| 11 | ## Uploading |
| 12 | |
| 13 | -The **Upload** panel on the Files page stores files or a folder. Choose files, choose a folder, or drop either onto the panel, and check **encrypt** to encrypt in the browser before sending. Plain uploads preserve filenames and selected folder paths. Uploading while inside a folder adds the selection there. **Import from URL** stores a copy of a public HTTPS URL or Blossom URI. |
| 14 | +The **Upload** panel on the Files page stores files or a folder. Choose files, choose a folder, or drop either onto the panel. **Who can open** controls access: **Anyone with the link** is public, while **Relay members** requires current membership. Public is the default. Uploading while inside a folder adds the selection there. **Import from URL** stores a copy of a public HTTPS URL or Blossom URI. |
| 15 | |
| 16 | -Blossom identifies file contents by hash. Tiny keeps plain upload names and paths with your upload record; those labels do not change the file's hash or travel automatically to another Blossom server. Older uploads use names from site manifests or room attachments where available. Otherwise, they show a file type and short hash. |
| 17 | +Use **Advanced** to enable **Encrypt with a secret link** for a public upload. The browser encrypts the contents before sending them, and only someone with the complete secret link can decrypt them. Secret-link encryption is unavailable when **Relay members** is selected because member access is enforced by the relay, which must be able to serve the file after checking membership. Member access is private access control, not end-to-end encryption at rest; the relay stores the uploaded bytes and trusts its membership check. |
| 18 | + |
| 19 | +Blossom identifies file contents by hash. Tiny sends a file's name, path, MIME type and access choice as upload metadata, then keeps those labels with your catalog entry. Metadata does not change the file's hash and does not travel automatically to another Blossom server. The same content hash cannot represent both a public file and a member-only file: uploading identical bytes again with a different access choice does not create a second policy-specific object. Older uploads use names from site manifests or room attachments where available. Otherwise, they appear in **Unorganized uploads** with a file type and short hash. |
| 20 | |
| 21 | ## Encrypted files |
| 22 | |
| 23 | -An encrypted single file gets a fresh AES-256-GCM key and nonce. The resulting share link keeps the key, nonce, original name and file type in its URL fragment. Keep that link: the relay cannot recover the key. Anyone who has the complete link and permission to download the blob can decrypt it. |
| 24 | +An encrypted single file gets a fresh AES-256-GCM key and nonce. The resulting share link keeps the key, nonce, original name and file type in its URL fragment. The browser saves the finished file's key and display metadata in local storage scoped to the current account and tenant, so the file can be recognized after a reload in that browser. Clearing browser storage removes that local copy. Keep the complete link for use on another browser or device; the relay cannot recover the key. |
| 25 | |
| 26 | Encrypted files remain subject to the tenant's download policy. Sending an encrypted link does not grant membership in a private tenant. Removing a file or revoking membership cannot recall copies someone has already downloaded. |
| 27 | |
| 28 | -Open a stored file to copy its Blossom URI or share link. Random-key files can also be sent through [NIP-17 file messages](https://github.com/nostr-protocol/nips/blob/master/17.md). Connect a signer with NIP-44 support and enter the recipient's public key. Both parties need signed kind 10050 inbox relay lists available to this relay. The browser sends encrypted gift wraps to those inbox relays, including a copy for the sender. It refuses delivery when the required lists are missing. |
| 29 | +Open a stored file to copy its Blossom URI or explicitly copy its secret link. The application does not put a secret link on the clipboard automatically. Random-key files can also be sent through [NIP-17 file messages](https://github.com/nostr-protocol/nips/blob/master/17.md). Connect a signer with NIP-44 support and enter the recipient's public key. Both parties need signed kind 10050 inbox relay lists available to this relay. The browser sends encrypted gift wraps to those inbox relays, including a copy for the sender. It refuses delivery when the required lists are missing. |
| 30 | |
| 31 | ## Folders and large files |
| 32 | |
| 33 | An encrypted folder, or an encrypted file larger than 64 MiB, is stored as encrypted manifests. Files, chunks and all parent manifests use deduplicated keys that follow the [BUD-15 proposal](https://github.com/hzrd149/blossom/pull/104): the key derives from the content, so identical files share storage. This also reveals when files are identical and permits guesses about predictable content. Names, child keys and file metadata appear only inside encrypted manifests. Keep the complete share link to browse the folder or download individual files. |
| 34 | |
| 35 | -Folders use [draft BUD-16](https://github.com/hzrd149/blossom/pull/105). Large files use [draft BUD-17](https://github.com/hzrd149/blossom/pull/106), with 2 MiB plaintext chunks and up to 174 links per manifest. Larger directories use nested manifests that appear as one directory in the browser. Every retrieved object is checked against its hash before decryption, and file downloads verify their declared sizes. |
| 36 | +Folders use [draft BUD-16](https://github.com/hzrd149/blossom/pull/105). Large files use [draft BUD-17](https://github.com/hzrd149/blossom/pull/106), with 2 MiB plaintext chunks and up to 174 links per manifest. Larger directories use nested manifests that appear as one directory in the browser. Every retrieved object is checked against its hash before decryption, and file downloads verify their declared sizes. Decrypted images display inline, and text files open as readable text. Large text previews show the first 256 KiB; downloads contain the complete file. HTML and SVG display as source text. Decrypted MP4, WebM and Ogg video files play in the browser when the browser supports the format. |
| 37 | |
| 38 | The browser accepts up to 256 MiB per file, 1 GiB per folder, 10,000 files and 32 path levels. Browser folder selection includes files and their paths; empty folders are omitted. These browser limits apply in addition to the tenant's storage allowances. |
| 39 | |
| 40 | -Cancel stops the current upload. Retry reuses the selected files while the page stays open. Completed chunks may remain in the Files inventory after an interrupted folder upload. Removing a root manifest does not remove its child blobs, which may be shared by other folders. |
| 41 | +Progress reports human-readable MiB values and a percentage. Cancel stops the current upload. Retry reuses the selected files while the page stays open. For an encrypted retry, completed chunks and the same sealed key are reused. Refreshing the page loses the retry selection, but finished encrypted file keys remain available in this browser until its storage is cleared. A completed encrypted chunk root is cataloged as one named file; chunk objects are hidden from **My files**. Older or unrecognized uploads appear under **Unorganized uploads** so they can be reviewed without being silently removed. Removing a root manifest does not remove its child blobs, which may be shared by other folders. |
| 42 | |
| 43 | ## Resumable uploads |
| 44 | |
| 45 | -Encrypted file uploads use [draft BUD-14](https://github.com/hzrd149/blossom/pull/102) when the server advertises it. Cancel and retry keep the encrypted bytes and key in memory while the page stays open. Closing or reloading the page loses that local state. The uploader falls back to BUD-13 when multipart support is unavailable. |
| 46 | +Encrypted file uploads use [draft BUD-14](https://github.com/hzrd149/blossom/pull/102) when the server advertises it. The browser retries transient failures automatically with backoff and keeps completed encrypted chunks and the same key for a retry on the current page. **Continue in the background** is an explicit opt-in when the browser supports Background Fetch; otherwise the upload remains in the foreground. The uploader falls back to BUD-13 when multipart support is unavailable. |
| 47 | |
| 48 | Other clients can send binary chunks to `PATCH /<sha256>` with `Upload-Type`, `Upload-Length`, `Upload-Offset` and `Content-Length`. Chunks may overlap or arrive out of order. A partial upload stays unavailable for download until its complete hash is verified. Other uploads can proceed during verification. Blossom proofs name the final hash; NIP-98 proofs bind each chunk's request body. |
| 49 | |
| 50 | @@ -49,7 +51,7 @@ Set byte allowances with an owner-authorized `setpolicy` patch: |
| 51 | } |
| 52 | ``` |
| 53 | |
| 54 | -This example limits each stored blob to 100 MiB and allows 1 GiB of claimed storage per uploader. Encrypted chunks and manifests each count toward storage. The per-blob limit applies to each chunk, rather than the reassembled size of a chunked file. Zero means unlimited. Changes apply to subsequent uploads; lowering an allowance does not delete existing files. |
| 55 | +This example limits each stored blob to 100 MiB (104,857,600 bytes) and allows 1 GiB of claimed storage per uploader. The Files page displays sizes in MiB; storage policies use exact byte counts. Encrypted chunks and manifests each count toward storage. The per-blob limit applies to each chunk, rather than the reassembled size of a chunked file. Zero means unlimited. Changes apply to subsequent uploads; lowering an allowance does not delete existing files. |
| 56 | |
| 57 | Each uploader pays the full size of every file they claim, even when another uploader already stores identical bytes. Repeated uploads by the same user count once. Removing an uploader's claim releases their allowance; shared content remains while another claim exists. An owner can remove the stored file for everyone. |
| 58 | |
| 59 | diff --git a/internal/blob/http.go b/internal/blob/http.go |
| 60 | index a4cbf86..9da91c3 100644 |
| 61 | --- a/internal/blob/http.go |
| 62 | +++ b/internal/blob/http.go |
| 63 | @@ -1,11 +1,13 @@ |
| 64 | package blob |
| 65 | |
| 66 | import ( |
| 67 | + "bytes" |
| 68 | "context" |
| 69 | "database/sql" |
| 70 | "encoding/json" |
| 71 | "errors" |
| 72 | "fmt" |
| 73 | + "io" |
| 74 | "net/http" |
| 75 | "os" |
| 76 | "path/filepath" |
| 77 | @@ -44,6 +46,10 @@ func (s *Service) serveHTTP(w http.ResponseWriter, r *http.Request) { |
| 78 | s.mirror(w, r) |
| 79 | return |
| 80 | } |
| 81 | + if r.URL.Path == "/files/metadata" && r.Method == http.MethodPost { |
| 82 | + s.metadata(w, r) |
| 83 | + return |
| 84 | + } |
| 85 | if match := blobPathPattern.FindStringSubmatch(r.URL.Path); match != nil && (r.Method == http.MethodPut || r.Method == http.MethodPatch || r.Method == http.MethodOptions) { |
| 86 | if r.Method == http.MethodPatch || r.Method == http.MethodOptions { |
| 87 | s.multipart(w, r, match[1]) |
| 88 | @@ -67,6 +73,43 @@ func (s *Service) serveHTTP(w http.ResponseWriter, r *http.Request) { |
| 89 | s.blob(w, r) |
| 90 | } |
| 91 | |
| 92 | +func (s *Service) metadata(w http.ResponseWriter, r *http.Request) { |
| 93 | + uploader, err := s.authorize(r, ActionUpload) |
| 94 | + if err != nil { |
| 95 | + s.fail(w, http.StatusUnauthorized, err) |
| 96 | + return |
| 97 | + } |
| 98 | + var input struct { |
| 99 | + Hash string `json:"hash"` |
| 100 | + Name string `json:"name"` |
| 101 | + Path string `json:"path"` |
| 102 | + Purpose string `json:"purpose"` |
| 103 | + Size int64 `json:"size"` |
| 104 | + Type string `json:"type"` |
| 105 | + } |
| 106 | + body, readErr := io.ReadAll(io.LimitReader(r.Body, 16*1024+1)) |
| 107 | + if readErr != nil || len(body) > 16*1024 { |
| 108 | + s.fail(w, http.StatusBadRequest, errors.New("invalid: metadata body")) |
| 109 | + return |
| 110 | + } |
| 111 | + decoder := json.NewDecoder(bytes.NewReader(body)) |
| 112 | + decoder.DisallowUnknownFields() |
| 113 | + if err := decoder.Decode(&input); err != nil { |
| 114 | + s.fail(w, http.StatusBadRequest, errors.New("invalid: metadata body")) |
| 115 | + return |
| 116 | + } |
| 117 | + var extra any |
| 118 | + if err := decoder.Decode(&extra); err != io.EOF { |
| 119 | + s.fail(w, http.StatusBadRequest, errors.New("invalid: metadata body")) |
| 120 | + return |
| 121 | + } |
| 122 | + if err := s.SetClaimMetadata(r.Context(), uploader, strings.ToLower(strings.TrimSpace(input.Hash)), input.Name, input.Path, input.Purpose, input.Type, input.Size); err != nil { |
| 123 | + s.fail(w, statusFor(err), err) |
| 124 | + return |
| 125 | + } |
| 126 | + s.writeJSON(w, http.StatusOK, map[string]any{"ok": true, "hash": strings.ToLower(strings.TrimSpace(input.Hash))}) |
| 127 | +} |
| 128 | + |
| 129 | func (s *Service) upload(w http.ResponseWriter, r *http.Request) { |
| 130 | pubkey, err := s.authorize(r, ActionUpload) |
| 131 | if err != nil { |
| 132 | @@ -83,7 +126,7 @@ func (s *Service) upload(w http.ResponseWriter, r *http.Request) { |
| 133 | s.fail(w, http.StatusBadRequest, err) |
| 134 | return |
| 135 | } |
| 136 | - blob, created, err := s.putValidatedWithMetadata(r.Context(), r.Body, contentType(r.Header.Get("content-type")), pubkey, claimed, name, metadataPath, func(actual string) error { |
| 137 | + blob, created, err := s.putValidatedWithMetadata(r.Context(), r.Body, contentType(r.Header.Get("content-type")), pubkey, claimed, name, metadataPath, r.URL.Query().Get("access"), r.URL.Query().Get("purpose"), func(actual string) error { |
| 138 | if s.config.ValidateUpload != nil { |
| 139 | return s.config.ValidateUpload(r, actual) |
| 140 | } |
| 141 | diff --git a/internal/blob/metadata.go b/internal/blob/metadata.go |
| 142 | index 714f742..91914e6 100644 |
| 143 | --- a/internal/blob/metadata.go |
| 144 | +++ b/internal/blob/metadata.go |
| 145 | @@ -20,6 +20,47 @@ var ( |
| 146 | errMismatchedClaimName = errors.New("invalid: filename must match the upload path basename") |
| 147 | ) |
| 148 | |
| 149 | +// SetClaimMetadata records logical file metadata without changing blob bytes. |
| 150 | +// It is used after an encrypted root is uploaded; decryption keys never enter |
| 151 | +// relay storage. The caller must already have authenticated the uploader. |
| 152 | +func (s *Service) SetClaimMetadata(ctx context.Context, uploader, sha, name, rawPath, purpose, typ string, size int64) error { |
| 153 | + if !shaPattern.MatchString(sha) || uploader == "" { |
| 154 | + return errors.New("invalid: blob metadata target") |
| 155 | + } |
| 156 | + if purpose != "file" && purpose != "chunk" { |
| 157 | + return errors.New("invalid: blob metadata purpose") |
| 158 | + } |
| 159 | + if size < 0 || size > 1<<40 { |
| 160 | + return errors.New("invalid: logical file size") |
| 161 | + } |
| 162 | + name, rawPath, err := claimMetadata(name, rawPath) |
| 163 | + if err != nil { |
| 164 | + return err |
| 165 | + } |
| 166 | + if purpose == "file" && name == "" { |
| 167 | + return errors.New("invalid: logical file name is required") |
| 168 | + } |
| 169 | + if typ == "" { |
| 170 | + typ = "application/octet-stream" |
| 171 | + } |
| 172 | + if len(typ) > 256 || !strings.Contains(typ, "/") || hasControl(typ) { |
| 173 | + return errors.New("invalid: logical file type") |
| 174 | + } |
| 175 | + return s.store.WithTx(ctx, func(tx *sql.Tx) error { |
| 176 | + var found int |
| 177 | + if err := tx.QueryRowContext(ctx, "SELECT 1 FROM blob_claims WHERE sha256=? AND uploader=?", sha, uploader).Scan(&found); err != nil { |
| 178 | + if errors.Is(err, sql.ErrNoRows) { |
| 179 | + return errors.New("not found: uploader blob claim") |
| 180 | + } |
| 181 | + return err |
| 182 | + } |
| 183 | + _, err := tx.ExecContext(ctx, `INSERT INTO blob_claim_metadata(sha256,uploader,name,path,purpose,logical_size,logical_type,updated_at) |
| 184 | + VALUES(?,?,?,?,?,?,?,?) ON CONFLICT(sha256,uploader,path) DO UPDATE SET name=excluded.name,purpose=excluded.purpose,logical_size=excluded.logical_size,logical_type=excluded.logical_type,updated_at=excluded.updated_at`, |
| 185 | + sha, uploader, name, rawPath, purpose, size, typ, nowUnix()) |
| 186 | + return err |
| 187 | + }) |
| 188 | +} |
| 189 | + |
| 190 | func validateClaimName(name string) error { |
| 191 | if name == "" { |
| 192 | return nil |
| 193 | @@ -77,13 +118,13 @@ func claimMetadata(name, rawPath string) (string, string, error) { |
| 194 | return name, cleanPath, nil |
| 195 | } |
| 196 | |
| 197 | -func (s *Service) saveClaimMetadata(ctx context.Context, tx *sql.Tx, sha, uploader, name, metadataPath string) error { |
| 198 | - if uploader == "" || (name == "" && metadataPath == "") { |
| 199 | +func (s *Service) saveClaimMetadata(ctx context.Context, tx *sql.Tx, sha, uploader, name, metadataPath, purpose string) error { |
| 200 | + if uploader == "" || (name == "" && metadataPath == "" && purpose == "") { |
| 201 | return nil |
| 202 | } |
| 203 | - _, err := tx.ExecContext(ctx, `INSERT INTO blob_claim_metadata(sha256,uploader,name,path,updated_at) |
| 204 | - VALUES(?,?,?,?,?) ON CONFLICT(sha256,uploader,path) DO UPDATE SET name=excluded.name,updated_at=excluded.updated_at`, |
| 205 | - sha, uploader, name, metadataPath, nowUnix()) |
| 206 | + _, err := tx.ExecContext(ctx, `INSERT INTO blob_claim_metadata(sha256,uploader,name,path,purpose,updated_at) |
| 207 | + VALUES(?,?,?,?,?,?) ON CONFLICT(sha256,uploader,path) DO UPDATE SET name=excluded.name,purpose=excluded.purpose,updated_at=excluded.updated_at`, |
| 208 | + sha, uploader, name, metadataPath, purpose, nowUnix()) |
| 209 | if err != nil { |
| 210 | return fmt.Errorf("record blob claim metadata: %w", err) |
| 211 | } |
| 212 | @@ -91,7 +132,7 @@ func (s *Service) saveClaimMetadata(ctx context.Context, tx *sql.Tx, sha, upload |
| 213 | } |
| 214 | |
| 215 | func (s *Service) ListClaimMetadata(ctx context.Context, uploader string) ([]BlobClaimMetadata, error) { |
| 216 | - rows, err := s.store.DB().QueryContext(ctx, `SELECT c.sha256,c.uploader,COALESCE(m.name,''),COALESCE(m.path,''),COALESCE(m.updated_at,c.claimed_at) |
| 217 | + rows, err := s.store.DB().QueryContext(ctx, `SELECT c.sha256,c.uploader,COALESCE(m.name,''),COALESCE(m.path,''),COALESCE(m.purpose,''),COALESCE(m.logical_size,0),COALESCE(m.logical_type,''),COALESCE(m.updated_at,c.claimed_at) |
| 218 | FROM blob_claims c LEFT JOIN blob_claim_metadata m ON m.sha256=c.sha256 AND m.uploader=c.uploader |
| 219 | JOIN blobs b ON b.sha256=c.sha256 WHERE c.uploader=? ORDER BY COALESCE(m.updated_at,c.claimed_at) DESC,c.sha256 DESC`, uploader) |
| 220 | if err != nil { |
| 221 | @@ -101,7 +142,7 @@ func (s *Service) ListClaimMetadata(ctx context.Context, uploader string) ([]Blo |
| 222 | entries := make([]BlobClaimMetadata, 0) |
| 223 | for rows.Next() { |
| 224 | var entry BlobClaimMetadata |
| 225 | - if err := rows.Scan(&entry.SHA256, &entry.Uploader, &entry.Name, &entry.Path, &entry.UpdatedAt); err != nil { |
| 226 | + if err := rows.Scan(&entry.SHA256, &entry.Uploader, &entry.Name, &entry.Path, &entry.Purpose, &entry.Size, &entry.Type, &entry.UpdatedAt); err != nil { |
| 227 | return nil, fmt.Errorf("scan blob claim metadata: %w", err) |
| 228 | } |
| 229 | entries = append(entries, entry) |
| 230 | diff --git a/internal/blob/metadata_test.go b/internal/blob/metadata_test.go |
| 231 | index a95064b..3b6959c 100644 |
| 232 | --- a/internal/blob/metadata_test.go |
| 233 | +++ b/internal/blob/metadata_test.go |
| 234 | @@ -31,6 +31,24 @@ func TestUploadStoresClaimScopedNameAndPath(t *testing.T) { |
| 235 | } |
| 236 | } |
| 237 | |
| 238 | +func TestBlobAccessIsDurableAndCannotChangeOnDeduplication(t *testing.T) { |
| 239 | + s := testService(t) |
| 240 | + ctx := context.Background() |
| 241 | + entry, err := s.Put(ctx, PutOptions{Reader: strings.NewReader("member bytes"), Type: "text/plain", Uploader: "uploader", Access: AccessMembers}) |
| 242 | + if err != nil { |
| 243 | + t.Fatal(err) |
| 244 | + } |
| 245 | + if access, err := s.BlobAccess(ctx, entry.SHA256); err != nil || access != AccessMembers { |
| 246 | + t.Fatalf("access = %q, %v", access, err) |
| 247 | + } |
| 248 | + if _, err := s.Put(ctx, PutOptions{Reader: strings.NewReader("member bytes"), Type: "text/plain", Uploader: "other", Access: AccessPublic}); !errors.Is(err, ErrAccessConflict) { |
| 249 | + t.Fatalf("public deduplication error = %v", err) |
| 250 | + } |
| 251 | + if access, err := s.BlobAccess(ctx, entry.SHA256); err != nil || access != AccessMembers { |
| 252 | + t.Fatalf("access changed after rejected deduplication = %q, %v", access, err) |
| 253 | + } |
| 254 | +} |
| 255 | + |
| 256 | func TestValidateClaimPath(t *testing.T) { |
| 257 | for _, path := range []string{"", "file.txt", "dir/file.txt"} { |
| 258 | if _, err := validateClaimPath(path); err != nil { |
| 259 | diff --git a/internal/blob/multipart.go b/internal/blob/multipart.go |
| 260 | index c4817f1..375c509 100644 |
| 261 | --- a/internal/blob/multipart.go |
| 262 | +++ b/internal/blob/multipart.go |
| 263 | @@ -29,8 +29,8 @@ const ( |
| 264 | ) |
| 265 | |
| 266 | type multipartRequest struct { |
| 267 | - id, hash, uploader, typ, name, metadataPath string |
| 268 | - length, offset, count int64 |
| 269 | + id, hash, uploader, typ, name, metadataPath, access, purpose string |
| 270 | + length, offset, count int64 |
| 271 | } |
| 272 | |
| 273 | type multipartError struct { |
| 274 | @@ -90,6 +90,14 @@ func parseMultipart(r *http.Request, hash, uploader string) (multipartRequest, e |
| 275 | return part, multipartFailure(http.StatusBadRequest, metadataErr.Error()) |
| 276 | } |
| 277 | part.name, part.metadataPath = name, metadataPath |
| 278 | + part.access, metadataErr = parseAccess(r.URL.Query().Get("access")) |
| 279 | + if metadataErr != nil { |
| 280 | + return part, multipartFailure(http.StatusBadRequest, metadataErr.Error()) |
| 281 | + } |
| 282 | + part.purpose = strings.TrimSpace(r.URL.Query().Get("purpose")) |
| 283 | + if part.purpose != "" && part.purpose != "chunk" && part.purpose != "file" { |
| 284 | + return part, multipartFailure(http.StatusBadRequest, "invalid upload purpose") |
| 285 | + } |
| 286 | if r.ContentLength < 0 { |
| 287 | return part, multipartFailure(411, "Content-Length is required") |
| 288 | } |
| 289 | @@ -174,12 +182,12 @@ func (s *Service) multipartAdmission(ctx context.Context, part multipartRequest) |
| 290 | return s.checkMultipartQuota(ctx, part, limits) |
| 291 | } |
| 292 | var length int64 |
| 293 | - var typ, name, metadataPath string |
| 294 | - err := s.store.DB().QueryRowContext(ctx, "SELECT length,type,name,path FROM multipart_uploads WHERE id=?", part.id).Scan(&length, &typ, &name, &metadataPath) |
| 295 | + var typ, name, metadataPath, access, purpose string |
| 296 | + err := s.store.DB().QueryRowContext(ctx, "SELECT length,type,name,path,access,purpose FROM multipart_uploads WHERE id=?", part.id).Scan(&length, &typ, &name, &metadataPath, &access, &purpose) |
| 297 | if err != nil && !errors.Is(err, sql.ErrNoRows) { |
| 298 | return err |
| 299 | } |
| 300 | - if err == nil && (length != part.length || typ != part.typ) { |
| 301 | + if err == nil && (length != part.length || typ != part.typ || access != part.access || (purpose != "" && purpose != part.purpose)) { |
| 302 | return ErrMultipartConflict |
| 303 | } |
| 304 | if err := s.checkMultipartQuota(ctx, part, limits); err != nil { |
| 305 | @@ -191,7 +199,10 @@ func (s *Service) multipartAdmission(ctx context.Context, part multipartRequest) |
| 306 | if part.name == "" && part.metadataPath == "" { |
| 307 | part.name, part.metadataPath = name, metadataPath |
| 308 | } |
| 309 | - _, err = s.store.DB().ExecContext(ctx, "UPDATE multipart_uploads SET last_seen=?,name=?,path=? WHERE id=?", time.Now().Unix(), part.name, part.metadataPath, part.id) |
| 310 | + if part.purpose == "" { |
| 311 | + part.purpose = purpose |
| 312 | + } |
| 313 | + _, err = s.store.DB().ExecContext(ctx, "UPDATE multipart_uploads SET last_seen=?,name=?,path=?,purpose=? WHERE id=?", time.Now().Unix(), part.name, part.metadataPath, part.purpose, part.id) |
| 314 | return err |
| 315 | } |
| 316 | |
| 317 | @@ -230,7 +241,7 @@ func (s *Service) createMultipart(ctx context.Context, part multipartRequest) er |
| 318 | return err |
| 319 | } |
| 320 | now := time.Now().Unix() |
| 321 | - _, err = s.store.DB().ExecContext(ctx, "INSERT INTO multipart_uploads(id,sha256,uploader,length,type,created,last_seen,name,path) VALUES(?,?,?,?,?,?,?,?,?)", part.id, part.hash, part.uploader, part.length, part.typ, now, now, part.name, part.metadataPath) |
| 322 | + _, err = s.store.DB().ExecContext(ctx, "INSERT INTO multipart_uploads(id,sha256,uploader,length,type,created,last_seen,name,path,access,purpose) VALUES(?,?,?,?,?,?,?,?,?,?,?)", part.id, part.hash, part.uploader, part.length, part.typ, now, now, part.name, part.metadataPath, part.access, part.purpose) |
| 323 | if err != nil { |
| 324 | _ = os.Remove(path) |
| 325 | } |
| 326 | @@ -306,14 +317,14 @@ func (s *Service) commitMultipart(ctx context.Context, part multipartRequest, ch |
| 327 | s.multipartMu.Unlock() |
| 328 | return Blob{}, false, false, admissionErr |
| 329 | } |
| 330 | - name, metadataPath := part.name, part.metadataPath |
| 331 | + name, metadataPath, purpose := part.name, part.metadataPath, part.purpose |
| 332 | if name == "" && metadataPath == "" { |
| 333 | - if err := s.store.DB().QueryRowContext(ctx, "SELECT name,path FROM multipart_uploads WHERE id=?", part.id).Scan(&name, &metadataPath); err != nil && !errors.Is(err, sql.ErrNoRows) { |
| 334 | + if err := s.store.DB().QueryRowContext(ctx, "SELECT name,path,purpose FROM multipart_uploads WHERE id=?", part.id).Scan(&name, &metadataPath, &purpose); err != nil && !errors.Is(err, sql.ErrNoRows) { |
| 335 | s.multipartMu.Unlock() |
| 336 | return Blob{}, false, false, err |
| 337 | } |
| 338 | } |
| 339 | - candidate := uploadCandidate{hash: part.hash, size: part.length, typ: part.typ, uploader: part.uploader, name: name, metadataPath: metadataPath, reservation: part.id} |
| 340 | + candidate := uploadCandidate{hash: part.hash, size: part.length, typ: part.typ, uploader: part.uploader, name: name, metadataPath: metadataPath, access: part.access, purpose: purpose, reservation: part.id} |
| 341 | if existing, err := s.lookup(ctx, part.hash); err == nil { |
| 342 | if existing.Size != part.length { |
| 343 | s.multipartMu.Unlock() |
| 344 | diff --git a/internal/blob/path_upload.go b/internal/blob/path_upload.go |
| 345 | index 35fabf3..649f77a 100644 |
| 346 | --- a/internal/blob/path_upload.go |
| 347 | +++ b/internal/blob/path_upload.go |
| 348 | @@ -25,14 +25,15 @@ func (s *Service) pathUpload(w http.ResponseWriter, r *http.Request, expectedHas |
| 349 | |
| 350 | urls := r.URL.Query()["url"] |
| 351 | if len(urls) == 0 { |
| 352 | - entry, created, putErr := s.putValidated(r.Context(), r.Body, |
| 353 | + entry, created, putErr := s.putValidatedWithMetadata(r.Context(), r.Body, |
| 354 | contentType(r.Header.Get("content-type")), pubkey, expectedHash, |
| 355 | + r.URL.Query().Get("filename"), r.URL.Query().Get("path"), r.URL.Query().Get("access"), r.URL.Query().Get("purpose"), |
| 356 | func(actual string) error { |
| 357 | if s.config.ValidateUpload != nil { |
| 358 | return s.config.ValidateUpload(r, actual) |
| 359 | } |
| 360 | return nil |
| 361 | - }) |
| 362 | + }, nil) |
| 363 | if putErr != nil { |
| 364 | s.fail(w, statusFor(putErr), putErr) |
| 365 | return |
| 366 | @@ -84,13 +85,13 @@ func (s *Service) pathUpload(w http.ResponseWriter, r *http.Request, expectedHas |
| 367 | if typ == "application/octet-stream" { |
| 368 | typ = typeForExtension(raw) |
| 369 | } |
| 370 | - entry, created, putErr := s.putValidated(r.Context(), response.Body, typ, |
| 371 | - pubkey, expectedHash, func(actual string) error { |
| 372 | + entry, created, putErr := s.putValidatedWithMetadata(r.Context(), response.Body, typ, |
| 373 | + pubkey, expectedHash, "", "", r.URL.Query().Get("access"), r.URL.Query().Get("purpose"), func(actual string) error { |
| 374 | if s.config.ValidateUpload != nil { |
| 375 | return s.config.ValidateUpload(r, actual) |
| 376 | } |
| 377 | return nil |
| 378 | - }) |
| 379 | + }, nil) |
| 380 | _ = response.Body.Close() |
| 381 | if putErr == nil { |
| 382 | s.writeJSON(w, chooseStatus(created, http.StatusCreated), s.descriptor(r, entry)) |
| 383 | diff --git a/internal/blob/path_upload_test.go b/internal/blob/path_upload_test.go |
| 384 | index a3f84d7..55e1d77 100644 |
| 385 | --- a/internal/blob/path_upload_test.go |
| 386 | +++ b/internal/blob/path_upload_test.go |
| 387 | @@ -129,3 +129,23 @@ func (r *trackingReader) Read(p []byte) (int, error) { |
| 388 | *r.read = true |
| 389 | return r.Reader.Read(p) |
| 390 | } |
| 391 | + |
| 392 | +func TestPathUploadPreservesAccessPurposeAndClaimMetadata(t *testing.T) { |
| 393 | + s := testService(t) |
| 394 | + body := "named member-only file" |
| 395 | + digest := sha256.Sum256([]byte(body)) |
| 396 | + hash := hex.EncodeToString(digest[:]) |
| 397 | + r := httptest.NewRequest(http.MethodPut, "https://relay.test/"+hash+"?filename=note.txt&path=docs%2Fnote.txt&access=members&purpose=file", strings.NewReader(body)) |
| 398 | + r.Header.Set("Content-Type", "text/plain") |
| 399 | + response := record(s.Handler(), r) |
| 400 | + if response.Code != http.StatusCreated { |
| 401 | + t.Fatalf("upload: %d %s", response.Code, response.Body.String()) |
| 402 | + } |
| 403 | + entries, err := s.ListClaimMetadata(t.Context(), "uploader") |
| 404 | + if err != nil || len(entries) != 1 || entries[0].Purpose != "file" || entries[0].Name != "note.txt" || entries[0].Path != "docs/note.txt" { |
| 405 | + t.Fatalf("claim: %+v %v", entries, err) |
| 406 | + } |
| 407 | + if access, err := s.BlobAccess(t.Context(), hash); err != nil || access != AccessMembers { |
| 408 | + t.Fatalf("access: %q %v", access, err) |
| 409 | + } |
| 410 | +} |
| 411 | diff --git a/internal/blob/service.go b/internal/blob/service.go |
| 412 | index 53bfc42..1506d2f 100644 |
| 413 | --- a/internal/blob/service.go |
| 414 | +++ b/internal/blob/service.go |
| 415 | @@ -105,6 +105,9 @@ type BlobClaimMetadata struct { |
| 416 | Uploader string `json:"uploader"` |
| 417 | Name string `json:"name"` |
| 418 | Path string `json:"path"` |
| 419 | + Purpose string `json:"purpose,omitempty"` |
| 420 | + Size int64 `json:"size,omitempty"` |
| 421 | + Type string `json:"type,omitempty"` |
| 422 | UpdatedAt int64 `json:"updated_at"` |
| 423 | } |
| 424 | |
| 425 | @@ -116,6 +119,10 @@ type PutOptions struct { |
| 426 | Hash string |
| 427 | Name string |
| 428 | Path string |
| 429 | + // Access controls who may retrieve the stored bytes. Empty means public. |
| 430 | + Access string |
| 431 | + // Purpose identifies logical encrypted roots or their internal chunks. |
| 432 | + Purpose string |
| 433 | // Commit records ownership metadata in the blob transaction. Its isNew |
| 434 | // argument reports whether the blob was newly installed. An error rolls |
| 435 | // back all claims. |
| 436 | @@ -149,6 +156,13 @@ var ErrFileTooLarge = errors.New("invalid: blob exceeds the per-file size limit" |
| 437 | var ErrQuotaExceeded = errors.New("restricted: uploader storage quota exceeded") |
| 438 | var ErrMultipartConflict = errors.New("conflict: upload metadata differs from existing session") |
| 439 | var ErrNotEncrypted = errors.New("restricted: agent grant requires encrypted uploads") |
| 440 | +var ErrAccessConflict = errors.New("conflict: the blob already has a different access policy") |
| 441 | +var ErrInvalidAccess = errors.New("invalid: access must be public or members") |
| 442 | + |
| 443 | +const ( |
| 444 | + AccessPublic = "public" |
| 445 | + AccessMembers = "members" |
| 446 | +) |
| 447 | |
| 448 | var shaPattern = regexp.MustCompile(`^[0-9a-f]{64}$`) |
| 449 | var blobPathPattern = regexp.MustCompile(`^/([0-9a-f]{64})(?:\.[a-z0-9]{1,8})?$`) |
| 450 | @@ -156,6 +170,7 @@ var blobPathPattern = regexp.MustCompile(`^/([0-9a-f]{64})(?:\.[a-z0-9]{1,8})?$` |
| 451 | // HandlesPath keeps the daemon mount in sync with every supported file door. |
| 452 | func HandlesPath(path string) bool { |
| 453 | return path == "/upload" || path == "/mirror" || path == "/report" || |
| 454 | + path == "/files/metadata" || |
| 455 | path == "/nip96" || path == "/.well-known/nostr/nip96.json" || |
| 456 | strings.HasPrefix(path, "/nip96/") || strings.HasPrefix(path, "/list/") || |
| 457 | blobPathPattern.MatchString(path) |
| 458 | @@ -177,7 +192,7 @@ func New(ctx context.Context, config Config) (*Service, error) { |
| 459 | } |
| 460 | if _, err := config.Store.DB().ExecContext(ctx, `CREATE TABLE IF NOT EXISTS blobs ( |
| 461 | sha256 TEXT PRIMARY KEY, size INTEGER NOT NULL, type TEXT NOT NULL, |
| 462 | - uploader TEXT NOT NULL, uploaded INTEGER NOT NULL |
| 463 | + uploader TEXT NOT NULL, uploaded INTEGER NOT NULL, access TEXT NOT NULL DEFAULT 'public' |
| 464 | ); CREATE TABLE IF NOT EXISTS blob_claims ( |
| 465 | sha256 TEXT NOT NULL, uploader TEXT NOT NULL, claimed_at INTEGER NOT NULL, |
| 466 | PRIMARY KEY (sha256, uploader), FOREIGN KEY (sha256) REFERENCES blobs(sha256) ON DELETE CASCADE |
| 467 | @@ -185,6 +200,7 @@ func New(ctx context.Context, config Config) (*Service, error) { |
| 468 | CREATE INDEX IF NOT EXISTS blob_claims_uploader ON blob_claims(uploader, claimed_at DESC, sha256 DESC); |
| 469 | CREATE TABLE IF NOT EXISTS blob_claim_metadata ( |
| 470 | sha256 TEXT NOT NULL, uploader TEXT NOT NULL, name TEXT NOT NULL, path TEXT NOT NULL, |
| 471 | + purpose TEXT NOT NULL DEFAULT '', logical_size INTEGER NOT NULL DEFAULT 0, logical_type TEXT NOT NULL DEFAULT '', |
| 472 | updated_at INTEGER NOT NULL, PRIMARY KEY (sha256,uploader,path), |
| 473 | FOREIGN KEY (sha256,uploader) REFERENCES blob_claims(sha256,uploader) ON DELETE CASCADE |
| 474 | ); |
| 475 | @@ -199,12 +215,20 @@ func New(ctx context.Context, config Config) (*Service, error) { |
| 476 | WHEN NEW.uploader != '' BEGIN INSERT OR IGNORE INTO blob_claims(sha256,uploader,claimed_at) VALUES(NEW.sha256,NEW.uploader,NEW.uploaded); END; |
| 477 | CREATE TABLE IF NOT EXISTS blob_blocks (sha256 TEXT PRIMARY KEY, reason TEXT NOT NULL DEFAULT '', blocked_at INTEGER NOT NULL DEFAULT 0); |
| 478 | CREATE TABLE IF NOT EXISTS blob_tombstones (sha256 TEXT PRIMARY KEY, deleted_at INTEGER NOT NULL); |
| 479 | - CREATE TABLE IF NOT EXISTS multipart_uploads (id TEXT PRIMARY KEY, sha256 TEXT NOT NULL, uploader TEXT NOT NULL, length INTEGER NOT NULL, type TEXT NOT NULL, created INTEGER NOT NULL, last_seen INTEGER NOT NULL, name TEXT NOT NULL DEFAULT '', path TEXT NOT NULL DEFAULT ''); |
| 480 | + CREATE TABLE IF NOT EXISTS multipart_uploads (id TEXT PRIMARY KEY, sha256 TEXT NOT NULL, uploader TEXT NOT NULL, length INTEGER NOT NULL, type TEXT NOT NULL, created INTEGER NOT NULL, last_seen INTEGER NOT NULL, name TEXT NOT NULL DEFAULT '', path TEXT NOT NULL DEFAULT '', access TEXT NOT NULL DEFAULT 'public', purpose TEXT NOT NULL DEFAULT ''); |
| 481 | CREATE TABLE IF NOT EXISTS multipart_parts (upload_id TEXT NOT NULL, offset INTEGER NOT NULL, length INTEGER NOT NULL, PRIMARY KEY(upload_id, offset), FOREIGN KEY(upload_id) REFERENCES multipart_uploads(id) ON DELETE CASCADE); |
| 482 | INSERT OR IGNORE INTO blob_claims(sha256,uploader,claimed_at) |
| 483 | SELECT sha256,uploader,uploaded FROM blobs WHERE uploader != ''`); err != nil { |
| 484 | return nil, fmt.Errorf("initialize blob metadata: %w", err) |
| 485 | } |
| 486 | + if _, err := config.Store.DB().ExecContext(ctx, "ALTER TABLE blobs ADD COLUMN access TEXT NOT NULL DEFAULT 'public'"); err != nil && !strings.Contains(err.Error(), "duplicate column") { |
| 487 | + return nil, fmt.Errorf("add blob access: %w", err) |
| 488 | + } |
| 489 | + for _, column := range []string{"purpose TEXT NOT NULL DEFAULT ''", "logical_size INTEGER NOT NULL DEFAULT 0", "logical_type TEXT NOT NULL DEFAULT ''"} { |
| 490 | + if _, err := config.Store.DB().ExecContext(ctx, "ALTER TABLE blob_claim_metadata ADD COLUMN "+column); err != nil && !strings.Contains(err.Error(), "duplicate column") { |
| 491 | + return nil, fmt.Errorf("add blob claim metadata field: %w", err) |
| 492 | + } |
| 493 | + } |
| 494 | // Claims made under an agent grant with a ttl carry an expiry; every |
| 495 | // earlier claim keeps 0, which never lapses. |
| 496 | if _, err := config.Store.DB().ExecContext(ctx, "ALTER TABLE blob_claims ADD COLUMN expires INTEGER NOT NULL DEFAULT 0"); err != nil && !strings.Contains(err.Error(), "duplicate column") { |
| 497 | @@ -215,6 +239,12 @@ func New(ctx context.Context, config Config) (*Service, error) { |
| 498 | return nil, fmt.Errorf("add multipart metadata: %w", err) |
| 499 | } |
| 500 | } |
| 501 | + if _, err := config.Store.DB().ExecContext(ctx, "ALTER TABLE multipart_uploads ADD COLUMN access TEXT NOT NULL DEFAULT 'public'"); err != nil && !strings.Contains(err.Error(), "duplicate column") { |
| 502 | + return nil, fmt.Errorf("add multipart access: %w", err) |
| 503 | + } |
| 504 | + if _, err := config.Store.DB().ExecContext(ctx, "ALTER TABLE multipart_uploads ADD COLUMN purpose TEXT NOT NULL DEFAULT ''"); err != nil && !strings.Contains(err.Error(), "duplicate column") { |
| 505 | + return nil, fmt.Errorf("add multipart purpose: %w", err) |
| 506 | + } |
| 507 | if _, err := config.Store.DB().ExecContext(ctx, "CREATE INDEX IF NOT EXISTS blob_claims_expires ON blob_claims(expires) WHERE expires>0"); err != nil { |
| 508 | return nil, fmt.Errorf("index blob claim expiry: %w", err) |
| 509 | } |
| 510 | diff --git a/internal/blob/storage.go b/internal/blob/storage.go |
| 511 | index 7f3a05b..d4fb336 100644 |
| 512 | --- a/internal/blob/storage.go |
| 513 | +++ b/internal/blob/storage.go |
| 514 | @@ -27,7 +27,7 @@ func (s *Service) Put(ctx context.Context, options PutOptions) (Blob, error) { |
| 515 | } |
| 516 | |
| 517 | func (s *Service) putOptions(ctx context.Context, options PutOptions) (Blob, bool, error) { |
| 518 | - return s.putValidatedWithMetadata(ctx, options.Reader, options.Type, options.Uploader, options.Hash, options.Name, options.Path, nil, options.Commit) |
| 519 | + return s.putValidatedWithMetadata(ctx, options.Reader, options.Type, options.Uploader, options.Hash, options.Name, options.Path, options.Access, options.Purpose, nil, options.Commit) |
| 520 | } |
| 521 | |
| 522 | // Get opens a verified metadata-backed blob for streaming to another service. |
| 523 | @@ -205,15 +205,23 @@ func (s *Service) put(ctx context.Context, source io.Reader, typ, uploader, clai |
| 524 | } |
| 525 | |
| 526 | func (s *Service) putValidated(ctx context.Context, source io.Reader, typ, uploader, claimed string, validate func(string) error) (Blob, bool, error) { |
| 527 | - return s.putValidatedWithMetadata(ctx, source, typ, uploader, claimed, "", "", validate, nil) |
| 528 | + return s.putValidatedWithMetadata(ctx, source, typ, uploader, claimed, "", "", "", "", validate, nil) |
| 529 | +} |
| 530 | + |
| 531 | +func (s *Service) putValidatedWithAccess(ctx context.Context, source io.Reader, typ, uploader, claimed, access string, validate func(string) error) (Blob, bool, error) { |
| 532 | + return s.putValidatedWithMetadata(ctx, source, typ, uploader, claimed, "", "", access, "", validate, nil) |
| 533 | } |
| 534 | |
| 535 | func (s *Service) putValidatedWithCommit(ctx context.Context, source io.Reader, typ, uploader, claimed string, validate func(string) error, commit func(context.Context, *sql.Tx, Blob, bool) error) (Blob, bool, error) { |
| 536 | - return s.putValidatedWithMetadata(ctx, source, typ, uploader, claimed, "", "", validate, commit) |
| 537 | + return s.putValidatedWithMetadata(ctx, source, typ, uploader, claimed, "", "", "", "", validate, commit) |
| 538 | } |
| 539 | |
| 540 | -func (s *Service) putValidatedWithMetadata(ctx context.Context, source io.Reader, typ, uploader, claimed, name, path string, validate func(string) error, commit func(context.Context, *sql.Tx, Blob, bool) error) (Blob, bool, error) { |
| 541 | +func (s *Service) putValidatedWithMetadata(ctx context.Context, source io.Reader, typ, uploader, claimed, name, path, access, purpose string, validate func(string) error, commit func(context.Context, *sql.Tx, Blob, bool) error) (Blob, bool, error) { |
| 542 | var err error |
| 543 | + access, err = parseAccess(access) |
| 544 | + if err != nil { |
| 545 | + return Blob{}, false, err |
| 546 | + } |
| 547 | name, path, err = claimMetadata(name, path) |
| 548 | if err != nil { |
| 549 | return Blob{}, false, err |
| 550 | @@ -264,17 +272,21 @@ func (s *Service) putValidatedWithMetadata(ctx context.Context, source io.Reader |
| 551 | return Blob{}, false, err |
| 552 | } |
| 553 | } |
| 554 | - return s.installUpload(ctx, uploadCandidate{path: tmpName, hash: sha, size: count, typ: typ, uploader: uploader, name: name, metadataPath: path, commit: commit}) |
| 555 | + return s.installUpload(ctx, uploadCandidate{path: tmpName, hash: sha, size: count, typ: typ, uploader: uploader, name: name, metadataPath: path, access: access, purpose: purpose, commit: commit}) |
| 556 | } |
| 557 | |
| 558 | type uploadCandidate struct { |
| 559 | - path, hash, typ, uploader, reservation, name, metadataPath string |
| 560 | - size int64 |
| 561 | - commit func(context.Context, *sql.Tx, Blob, bool) error |
| 562 | + path, hash, typ, uploader, reservation, name, metadataPath, access, purpose string |
| 563 | + size int64 |
| 564 | + commit func(context.Context, *sql.Tx, Blob, bool) error |
| 565 | } |
| 566 | |
| 567 | func (s *Service) installUpload(ctx context.Context, candidate uploadCandidate) (Blob, bool, error) { |
| 568 | sha, count, typ, uploader := candidate.hash, candidate.size, candidate.typ, candidate.uploader |
| 569 | + access, accessErr := parseAccess(candidate.access) |
| 570 | + if accessErr != nil { |
| 571 | + return Blob{}, false, accessErr |
| 572 | + } |
| 573 | // Serialize the lookup, quota check, installation and metadata transaction. |
| 574 | // This closes the race where concurrent uploads could each observe the same |
| 575 | // remaining quota before one of them installs a new physical object. |
| 576 | @@ -295,6 +307,13 @@ func (s *Service) installUpload(ctx context.Context, candidate uploadCandidate) |
| 577 | return Blob{}, false, ErrQuotaExceeded |
| 578 | } |
| 579 | if existing, err := s.lookup(ctx, sha); err == nil { |
| 580 | + existingAccess, accessErr := s.BlobAccess(ctx, sha) |
| 581 | + if accessErr != nil { |
| 582 | + return Blob{}, false, accessErr |
| 583 | + } |
| 584 | + if existingAccess != access { |
| 585 | + return Blob{}, false, ErrAccessConflict |
| 586 | + } |
| 587 | if terms.Encrypted && !encryptedTypes[existing.Type] { |
| 588 | return Blob{}, false, ErrNotEncrypted |
| 589 | } |
| 590 | @@ -319,7 +338,7 @@ func (s *Service) installUpload(ctx context.Context, candidate uploadCandidate) |
| 591 | if _, err := tx.ExecContext(ctx, claimSQL, sha, uploader, time.Now().Unix(), terms.ExpiresAt); err != nil { |
| 592 | return err |
| 593 | } |
| 594 | - if err := s.saveClaimMetadata(ctx, tx, sha, uploader, candidate.name, candidate.metadataPath); err != nil { |
| 595 | + if err := s.saveClaimMetadata(ctx, tx, sha, uploader, candidate.name, candidate.metadataPath, candidate.purpose); err != nil { |
| 596 | return err |
| 597 | } |
| 598 | } |
| 599 | @@ -385,14 +404,14 @@ func (s *Service) installUpload(ctx context.Context, candidate uploadCandidate) |
| 600 | if _, err := tx.ExecContext(ctx, "DELETE FROM blob_tombstones WHERE sha256=?", entry.SHA256); err != nil { |
| 601 | return fmt.Errorf("clear blob deletion tombstone: %w", err) |
| 602 | } |
| 603 | - if _, err := tx.ExecContext(ctx, "INSERT OR IGNORE INTO blobs(sha256,size,type,uploader,uploaded) VALUES(?,?,?,?,?)", entry.SHA256, entry.Size, entry.Type, entry.Uploader, entry.Uploaded); err != nil { |
| 604 | + if _, err := tx.ExecContext(ctx, "INSERT OR IGNORE INTO blobs(sha256,size,type,uploader,uploaded,access) VALUES(?,?,?,?,?,?)", entry.SHA256, entry.Size, entry.Type, entry.Uploader, entry.Uploaded, access); err != nil { |
| 605 | return fmt.Errorf("record blob: %w", err) |
| 606 | } |
| 607 | if entry.Uploader != "" { |
| 608 | if _, err := tx.ExecContext(ctx, claimSQL, entry.SHA256, entry.Uploader, entry.Uploaded, terms.ExpiresAt); err != nil { |
| 609 | return fmt.Errorf("record blob claim: %w", err) |
| 610 | } |
| 611 | - if err := s.saveClaimMetadata(ctx, tx, entry.SHA256, entry.Uploader, candidate.name, candidate.metadataPath); err != nil { |
| 612 | + if err := s.saveClaimMetadata(ctx, tx, entry.SHA256, entry.Uploader, candidate.name, candidate.metadataPath, candidate.purpose); err != nil { |
| 613 | return err |
| 614 | } |
| 615 | } |
| 616 | @@ -531,6 +550,32 @@ func (s *Service) lookup(ctx context.Context, sha string) (Blob, error) { |
| 617 | return entry, err |
| 618 | } |
| 619 | |
| 620 | +// BlobAccess returns the durable retrieval policy for a blob. Older databases |
| 621 | +// are treated as public through the column default and NULL-safe fallback. |
| 622 | +func (s *Service) BlobAccess(ctx context.Context, sha string) (string, error) { |
| 623 | + var access string |
| 624 | + err := s.store.DB().QueryRowContext(ctx, "SELECT COALESCE(access,?) FROM blobs WHERE sha256=?", AccessPublic, sha).Scan(&access) |
| 625 | + if err != nil { |
| 626 | + return "", err |
| 627 | + } |
| 628 | + parsed, parseErr := parseAccess(access) |
| 629 | + if parseErr != nil { |
| 630 | + return "", parseErr |
| 631 | + } |
| 632 | + return parsed, nil |
| 633 | +} |
| 634 | + |
| 635 | +func parseAccess(access string) (string, error) { |
| 636 | + access = strings.ToLower(strings.TrimSpace(access)) |
| 637 | + if access == "" || access == AccessPublic { |
| 638 | + return AccessPublic, nil |
| 639 | + } |
| 640 | + if access == AccessMembers { |
| 641 | + return AccessMembers, nil |
| 642 | + } |
| 643 | + return "", ErrInvalidAccess |
| 644 | +} |
| 645 | + |
| 646 | func (s *Service) blocked(ctx context.Context, sha string) bool { |
| 647 | var found int |
| 648 | err := s.store.DB().QueryRowContext(ctx, `SELECT 1 FROM hidden_events WHERE id=? |
| 649 | diff --git a/internal/daemon/client_browse.go b/internal/daemon/client_browse.go |
| 650 | index a2d80b7..e7a85a6 100644 |
| 651 | --- a/internal/daemon/client_browse.go |
| 652 | +++ b/internal/daemon/client_browse.go |
| 653 | @@ -240,6 +240,9 @@ func (t *Tenant) browseFile(ctx context.Context, actor, hash string) (any, error |
| 654 | if !t.Policy().Features.Files { |
| 655 | return nil, errors.New("not found: files are disabled") |
| 656 | } |
| 657 | + if err := t.fileReadAccess(ctx, actor, hash); err != nil { |
| 658 | + return nil, err |
| 659 | + } |
| 660 | if err := t.roomAttachmentAccess(ctx, actor, hash); err != nil { |
| 661 | return nil, err |
| 662 | } |
| 663 | @@ -258,9 +261,18 @@ func (t *Tenant) browseFile(ctx context.Context, actor, hash string) (any, error |
| 664 | } |
| 665 | binary := bytes.IndexByte(data, 0) >= 0 || (!utf8.Valid(data) && !truncated) |
| 666 | result := t.browseBlobMetadata(entry) |
| 667 | + if access, accessErr := t.blobs.BlobAccess(ctx, entry.SHA256); accessErr == nil { |
| 668 | + result["access"] = access |
| 669 | + } |
| 670 | if actor != "" { |
| 671 | var name, filePath string |
| 672 | - if err := t.store.DB().QueryRowContext(ctx, `SELECT name,path FROM blob_claim_metadata WHERE sha256=? AND uploader=? ORDER BY updated_at DESC,path LIMIT 1`, entry.SHA256, actor).Scan(&name, &filePath); err == nil { |
| 673 | + metadataQuery := `SELECT name,path FROM blob_claim_metadata WHERE sha256=? AND uploader=? ORDER BY updated_at DESC,path LIMIT 1` |
| 674 | + metadataArgs := []any{entry.SHA256, actor} |
| 675 | + if access, accessErr := t.blobs.BlobAccess(ctx, entry.SHA256); accessErr == nil && access == blob.AccessMembers { |
| 676 | + metadataQuery = `SELECT name,path FROM blob_claim_metadata WHERE sha256=? ORDER BY updated_at DESC,path LIMIT 1` |
| 677 | + metadataArgs = []any{entry.SHA256} |
| 678 | + } |
| 679 | + if err := t.store.DB().QueryRowContext(ctx, metadataQuery, metadataArgs...).Scan(&name, &filePath); err == nil { |
| 680 | if name != "" { |
| 681 | result["name"] = name |
| 682 | } |
| 683 | diff --git a/internal/daemon/client_browse_http.go b/internal/daemon/client_browse_http.go |
| 684 | index 2af94db..99836a3 100644 |
| 685 | --- a/internal/daemon/client_browse_http.go |
| 686 | +++ b/internal/daemon/client_browse_http.go |
| 687 | @@ -99,6 +99,9 @@ func (t *Tenant) fileDownload(w http.ResponseWriter, r *http.Request, actor stri |
| 688 | return os.ErrNotExist |
| 689 | } |
| 690 | hash := r.URL.Query().Get("hash") |
| 691 | + if err := t.fileReadAccess(r.Context(), actor, hash); err != nil { |
| 692 | + return err |
| 693 | + } |
| 694 | if err := t.roomAttachmentAccess(r.Context(), actor, hash); err != nil { |
| 695 | return err |
| 696 | } |
| 697 | diff --git a/internal/daemon/client_browse_test.go b/internal/daemon/client_browse_test.go |
| 698 | index a92b59e..761d15d 100644 |
| 699 | --- a/internal/daemon/client_browse_test.go |
| 700 | +++ b/internal/daemon/client_browse_test.go |
| 701 | @@ -131,3 +131,21 @@ func TestOwnerFileSearchUsesLiteralTextBeforePagination(t *testing.T) { |
| 702 | t.Fatalf("literal search = %#v", row) |
| 703 | } |
| 704 | } |
| 705 | + |
| 706 | +func TestLibraryRowsGroupsLegacyUnnamedBinaryAndHidesMarkedChunks(t *testing.T) { |
| 707 | + rows := []map[string]any{ |
| 708 | + {"sha256": strings.Repeat("a", 64), "name": "", "path": "", "type": "application/octet-stream"}, |
| 709 | + {"sha256": strings.Repeat("b", 64), "name": "", "path": "", "type": "application/octet-stream", "purpose": "chunk"}, |
| 710 | + {"sha256": strings.Repeat("c", 64), "name": "notes.txt", "path": "notes.txt", "type": "text/plain"}, |
| 711 | + } |
| 712 | + got := libraryRows(rows, nil) |
| 713 | + if len(got) != 2 { |
| 714 | + t.Fatalf("library rows = %d, want 2", len(got)) |
| 715 | + } |
| 716 | + if got[0]["path"] != "Unorganized uploads/aaaaaaaaaaaa" || got[0]["name"] != "aaaaaaaaaaaa" { |
| 717 | + t.Fatalf("legacy row = %#v", got[0]) |
| 718 | + } |
| 719 | + if got[1]["path"] != "notes.txt" { |
| 720 | + t.Fatalf("named row changed = %#v", got[1]) |
| 721 | + } |
| 722 | +} |
| 723 | diff --git a/internal/daemon/client_files.go b/internal/daemon/client_files.go |
| 724 | index bae3fbf..42dd8b3 100644 |
| 725 | --- a/internal/daemon/client_files.go |
| 726 | +++ b/internal/daemon/client_files.go |
| 727 | @@ -59,7 +59,7 @@ func (t *Tenant) browseFiles(ctx context.Context, actor string, q clientBrowseRe |
| 728 | return result, nil |
| 729 | } |
| 730 | |
| 731 | -func (t Tenant) browseFilesLegacy(ctx context.Context, actor string, q clientBrowseRequest) (any, error) { |
| 732 | +func (t *Tenant) browseFilesLegacy(ctx context.Context, actor string, q clientBrowseRequest) (any, error) { |
| 733 | if actor == "" { |
| 734 | return nil, errors.New("auth-required: sign in to browse your files") |
| 735 | } |
| 736 | @@ -104,7 +104,7 @@ func (t Tenant) browseFilesLegacy(ctx context.Context, actor string, q clientBro |
| 737 | return map[string]any{"items": items, "next_cursor": next}, nil |
| 738 | } |
| 739 | |
| 740 | -func (t Tenant) browseAllFiles(ctx context.Context, q clientBrowseRequest) ([]blob.Blob, string, error) { |
| 741 | +func (t *Tenant) browseAllFiles(ctx context.Context, q clientBrowseRequest) ([]blob.Blob, string, error) { |
| 742 | query := "SELECT sha256,size,type,uploader,uploaded FROM blobs WHERE sha256>?" |
| 743 | args := []any{q.Cursor} |
| 744 | needle := strings.ToLower(strings.TrimSpace(q.Query)) |
| 745 | @@ -141,7 +141,7 @@ func blobMatchesQuery(e blob.Blob, q string) bool { |
| 746 | n := strings.ToLower(strings.TrimSpace(q)) |
| 747 | return n == "" || strings.Contains(strings.ToLower(e.SHA256), n) || strings.Contains(strings.ToLower(e.Type), n) || strings.Contains(strings.ToLower(e.Uploader), n) |
| 748 | } |
| 749 | -func (t Tenant) browseBlobMetadata(e blob.Blob) map[string]any { |
| 750 | +func (t *Tenant) browseBlobMetadata(e blob.Blob) map[string]any { |
| 751 | portable := blob.BlossomURI{Hash: e.SHA256, Extension: "bin", Author: e.Uploader, Size: e.Size} |
| 752 | if origin, err := url.Parse(t.publicURL); err == nil && strings.Trim(origin.Path, "/") == "" { |
| 753 | portable.Servers = []string{origin.Scheme + "://" + origin.Host} |
| 754 | @@ -149,12 +149,21 @@ func (t Tenant) browseBlobMetadata(e blob.Blob) map[string]any { |
| 755 | return map[string]any{"sha256": e.SHA256, "size": e.Size, "type": e.Type, "uploader": e.Uploader, "uploaded": e.Uploaded, "url": strings.TrimRight(t.publicURL, "/") + "/" + e.SHA256, "download_url": strings.TrimRight(t.publicURL, "/") + "/files/raw?hash=" + url.QueryEscape(e.SHA256), "blossom_uri": portable.String()} |
| 756 | } |
| 757 | |
| 758 | -func (t Tenant) claimRows(ctx context.Context, actor string, all bool) ([]map[string]any, error) { |
| 759 | - q := `SELECT b.sha256,b.size,b.type,c.uploader,b.uploaded,COALESCE(m.name,''),COALESCE(m.path,'') FROM blobs b JOIN blob_claims c ON c.sha256=b.sha256 LEFT JOIN blob_claim_metadata m ON m.sha256=b.sha256 AND m.uploader=c.uploader` |
| 760 | +func (t *Tenant) claimRows(ctx context.Context, actor string, all bool) ([]map[string]any, error) { |
| 761 | + q := `SELECT b.sha256,b.size,b.type,c.uploader,b.uploaded,COALESCE(m.name,''),COALESCE(m.path,''),COALESCE(m.purpose,''),COALESCE(m.logical_size,0),COALESCE(m.logical_type,''),b.access FROM blobs b JOIN blob_claims c ON c.sha256=b.sha256 LEFT JOIN blob_claim_metadata m ON m.sha256=b.sha256 AND m.uploader=c.uploader` |
| 762 | args := []any{} |
| 763 | if !all { |
| 764 | - q += " WHERE c.uploader=?" |
| 765 | - args = append(args, actor) |
| 766 | + includeMembers := false |
| 767 | + if role, roleErr := t.community.Role(ctx, actor); roleErr == nil && t.memberBlobReader(ctx, actor, role) { |
| 768 | + includeMembers = true |
| 769 | + } |
| 770 | + if includeMembers { |
| 771 | + q += " WHERE c.uploader=? OR b.access=?" |
| 772 | + args = append(args, actor, blob.AccessMembers) |
| 773 | + } else { |
| 774 | + q += " WHERE c.uploader=?" |
| 775 | + args = append(args, actor) |
| 776 | + } |
| 777 | } |
| 778 | q += " ORDER BY b.uploaded DESC,b.sha256 DESC" |
| 779 | rows, err := t.store.DB().QueryContext(ctx, q, args...) |
| 780 | @@ -165,19 +174,28 @@ func (t Tenant) claimRows(ctx context.Context, actor string, all bool) ([]map[st |
| 781 | out := []map[string]any{} |
| 782 | for rows.Next() { |
| 783 | var e blob.Blob |
| 784 | - var name, p string |
| 785 | - if err := rows.Scan(&e.SHA256, &e.Size, &e.Type, &e.Uploader, &e.Uploaded, &name, &p); err != nil { |
| 786 | + var name, p, purpose, logicalType, access string |
| 787 | + var logicalSize int64 |
| 788 | + if err := rows.Scan(&e.SHA256, &e.Size, &e.Type, &e.Uploader, &e.Uploaded, &name, &p, &purpose, &logicalSize, &logicalType, &access); err != nil { |
| 789 | return nil, err |
| 790 | } |
| 791 | m := t.browseBlobMetadata(e) |
| 792 | + m["access"] = access |
| 793 | m["name"] = name |
| 794 | m["path"] = p |
| 795 | + m["purpose"] = purpose |
| 796 | + if purpose == "file" && logicalType != "" { |
| 797 | + m["logical_size"] = logicalSize |
| 798 | + } |
| 799 | + if logicalType != "" { |
| 800 | + m["logical_type"] = logicalType |
| 801 | + } |
| 802 | out = append(out, m) |
| 803 | } |
| 804 | return out, rows.Err() |
| 805 | } |
| 806 | |
| 807 | -func (t Tenant) browseLibrary(ctx context.Context, actor string, q clientBrowseRequest) (any, error) { |
| 808 | +func (t *Tenant) browseLibrary(ctx context.Context, actor string, q clientBrowseRequest) (any, error) { |
| 809 | rows, err := t.claimRows(ctx, actor, false) |
| 810 | if err != nil { |
| 811 | return nil, err |
| 812 | @@ -193,9 +211,43 @@ func (t Tenant) browseLibrary(ctx context.Context, actor string, q clientBrowseR |
| 813 | for h := range rooms { |
| 814 | excluded[h] = true |
| 815 | } |
| 816 | + rows = libraryRows(rows, excluded) |
| 817 | return t.folderItems(rows, excluded, q, "library", false), nil |
| 818 | } |
| 819 | -func (t Tenant) folderItems(rows []map[string]any, excluded map[string]bool, q clientBrowseRequest, view string, includeExcluded bool) map[string]any { |
| 820 | + |
| 821 | +// libraryRows keeps implementation objects out of the user's file library. |
| 822 | +// New encrypted uploads mark their chunk claims with purpose=chunk. Older |
| 823 | +// unnamed octet streams cannot be identified safely from size alone, so they |
| 824 | +// remain visible in one unorganized folder instead of being deleted or |
| 825 | +// silently hidden. |
| 826 | +func libraryRows(rows []map[string]any, excluded map[string]bool) []map[string]any { |
| 827 | + out := make([]map[string]any, 0, len(rows)) |
| 828 | + for _, row := range rows { |
| 829 | + if row["purpose"] == "chunk" { |
| 830 | + continue |
| 831 | + } |
| 832 | + if row["purpose"] == "file" { |
| 833 | + if size, ok := row["logical_size"]; ok { |
| 834 | + row["size"] = size |
| 835 | + } |
| 836 | + } |
| 837 | + if excluded[row["sha256"].(string)] && row["name"] == "" { |
| 838 | + continue |
| 839 | + } |
| 840 | + if row["name"] == "" && row["path"] == "" && row["type"] == "application/octet-stream" { |
| 841 | + hash := row["sha256"].(string) |
| 842 | + short := hash |
| 843 | + if len(short) > 12 { |
| 844 | + short = short[:12] |
| 845 | + } |
| 846 | + row["path"] = "Unorganized uploads/" + short |
| 847 | + row["name"] = short |
| 848 | + } |
| 849 | + out = append(out, row) |
| 850 | + } |
| 851 | + return out |
| 852 | +} |
| 853 | +func (t *Tenant) folderItems(rows []map[string]any, excluded map[string]bool, q clientBrowseRequest, view string, includeExcluded bool) map[string]any { |
| 854 | prefix := strings.Trim(q.Path, "/") |
| 855 | if prefix != "" { |
| 856 | prefix += "/" |
| 857 | @@ -290,12 +342,12 @@ func breadcrumbs(view, p string) []map[string]string { |
| 858 | return out |
| 859 | } |
| 860 | |
| 861 | -func (t Tenant) contextHashes(ctx context.Context, actor string) (map[string][]string, map[string][]string, error) { |
| 862 | +func (t *Tenant) contextHashes(ctx context.Context, actor string) (map[string][]string, map[string][]string, error) { |
| 863 | sitesH := map[string][]string{} |
| 864 | roomsH := map[string][]string{} |
| 865 | var before *storage.EventCursor |
| 866 | for { |
| 867 | - r, err := t.store.Query(ctx, event.Filter{Kinds: []int{sites.KindSite, sites.KindNamedSite, sites.KindSiteSnapshot}}, storage.QueryOptions{Now: time.Now().Unix(), Access: storage.Access{PubKeys: browseSession(&t, actor).PubKeys}, Limit: 100, Before: before}) |
| 868 | + r, err := t.store.Query(ctx, event.Filter{Kinds: []int{sites.KindSite, sites.KindNamedSite, sites.KindSiteSnapshot}}, storage.QueryOptions{Now: time.Now().Unix(), Access: storage.Access{PubKeys: browseSession(t, actor).PubKeys}, Limit: 100, Before: before}) |
| 869 | if err != nil { |
| 870 | return nil, nil, err |
| 871 | } |
| 872 | @@ -342,7 +394,7 @@ func (t Tenant) contextHashes(ctx context.Context, actor string) (map[string][]s |
| 873 | return sitesH, roomsH, nil |
| 874 | } |
| 875 | |
| 876 | -func (t Tenant) browseStorage(ctx context.Context, actor string, q clientBrowseRequest) (any, error) { |
| 877 | +func (t *Tenant) browseStorage(ctx context.Context, actor string, q clientBrowseRequest) (any, error) { |
| 878 | query := `SELECT sha256,size,type,uploader,uploaded FROM blobs WHERE 1=1` |
| 879 | args := []any{} |
| 880 | if q.Cursor != "" { |
| 881 | @@ -393,7 +445,7 @@ func (t Tenant) browseStorage(ctx context.Context, actor string, q clientBrowseR |
| 882 | } |
| 883 | return map[string]any{"items": items, "view": "storage", "path": "", "breadcrumbs": breadcrumbs("storage", ""), "next_cursor": next}, nil |
| 884 | } |
| 885 | -func (t Tenant) browseRoomFiles(ctx context.Context, actor string, q clientBrowseRequest) (any, error) { |
| 886 | +func (t *Tenant) browseRoomFiles(ctx context.Context, actor string, q clientBrowseRequest) (any, error) { |
| 887 | _, rooms, err := t.contextHashes(ctx, actor) |
| 888 | if err != nil { |
| 889 | return nil, err |
| 890 | @@ -427,7 +479,7 @@ func (t Tenant) browseRoomFiles(ctx context.Context, actor string, q clientBrows |
| 891 | } |
| 892 | return result, nil |
| 893 | } |
| 894 | -func (t Tenant) browseSiteFiles(ctx context.Context, actor string, q clientBrowseRequest) (any, error) { |
| 895 | +func (t *Tenant) browseSiteFiles(ctx context.Context, actor string, q clientBrowseRequest) (any, error) { |
| 896 | sitesH, _, err := t.contextHashes(ctx, actor) |
| 897 | if err != nil { |
| 898 | return nil, err |
| 899 | @@ -447,7 +499,7 @@ func (t Tenant) browseSiteFiles(ctx context.Context, actor string, q clientBrows |
| 900 | |
| 901 | // Read only the referenced blobs. Claim labels belong to uploaders and must |
| 902 | // not become names for files viewed through someone else's site or room. |
| 903 | -func (t Tenant) contextFileRows(ctx context.Context, refs map[string][]string, contextName string) ([]map[string]any, error) { |
| 904 | +func (t *Tenant) contextFileRows(ctx context.Context, refs map[string][]string, contextName string) ([]map[string]any, error) { |
| 905 | hashes := make([]string, 0, len(refs)) |
| 906 | for hash := range refs { |
| 907 | hashes = append(hashes, hash) |
| 908 | diff --git a/internal/daemon/client_files_test.go b/internal/daemon/client_files_test.go |
| 909 | index 7895dd2..f2fdcbb 100644 |
| 910 | --- a/internal/daemon/client_files_test.go |
| 911 | +++ b/internal/daemon/client_files_test.go |
| 912 | @@ -33,12 +33,12 @@ func TestFolderItemsKeepsFoldersAndFilesPageable(t *testing.T) { |
| 913 | {"sha256": "aaaaaaaa", "type": "text/plain", "uploader": "alice", "uploaded": int64(2), "name": "readme.txt", "path": "docs/readme.txt"}, |
| 914 | {"sha256": "bbbbbbbb", "type": "image/png", "uploader": "alice", "uploaded": int64(1), "name": "cover.png", "path": "cover.png"}, |
| 915 | } |
| 916 | - first := (Tenant{}).folderItems(rows, map[string]bool{}, clientBrowseRequest{BrowseRequest: browseRequestForTest(1)}, "library", true) |
| 917 | + first := (&Tenant{}).folderItems(rows, map[string]bool{}, clientBrowseRequest{BrowseRequest: browseRequestForTest(1)}, "library", true) |
| 918 | items := first["items"].([]map[string]any) |
| 919 | if len(items) != 1 || items[0]["kind"] != "folder" || first["next_cursor"] == "" { |
| 920 | t.Fatalf("first page = %#v", first) |
| 921 | } |
| 922 | - second := (Tenant{}).folderItems(rows, map[string]bool{}, clientBrowseRequest{BrowseRequest: browseRequestForTest(1), Cursor: first["next_cursor"].(string)}, "library", true) |
| 923 | + second := (&Tenant{}).folderItems(rows, map[string]bool{}, clientBrowseRequest{BrowseRequest: browseRequestForTest(1), Cursor: first["next_cursor"].(string)}, "library", true) |
| 924 | items = second["items"].([]map[string]any) |
| 925 | if len(items) != 1 || items[0]["kind"] != "file" || items[0]["name"] != "cover.png" { |
| 926 | t.Fatalf("second page = %#v", second) |
| 927 | diff --git a/internal/daemon/file_metadata_test.go b/internal/daemon/file_metadata_test.go |
| 928 | new file mode 100644 |
| 929 | index 0000000..c000090 |
| 930 | --- /dev/null |
| 931 | +++ b/internal/daemon/file_metadata_test.go |
| 932 | @@ -0,0 +1,121 @@ |
| 933 | +package daemon |
| 934 | + |
| 935 | +import ( |
| 936 | + "encoding/json" |
| 937 | + "net/http" |
| 938 | + "net/http/httptest" |
| 939 | + "strings" |
| 940 | + "testing" |
| 941 | + |
| 942 | + "github.com/FelineStateMachine/tinyrelay/internal/blob" |
| 943 | +) |
| 944 | + |
| 945 | +func TestEncryptedFileMetadataEndpointBindsBodyAndClaim(t *testing.T) { |
| 946 | + _, tenant := testTenant(t) |
| 947 | + ctx := t.Context() |
| 948 | + entry, err := tenant.blobs.Put(ctx, blob.PutOptions{Reader: strings.NewReader("ciphertext"), Type: "application/octet-stream", Uploader: tenant.Policy().Owner}) |
| 949 | + if err != nil { |
| 950 | + t.Fatal(err) |
| 951 | + } |
| 952 | + body := `{"hash":"` + entry.SHA256 + `","name":"Encrypted file ` + entry.SHA256[:12] + `","path":"Encrypted file ` + entry.SHA256[:12] + `","purpose":"file","size":104857600,"type":"application/octet-stream"}` |
| 953 | + r := httptest.NewRequest(http.MethodPost, "http://relay.test/files/metadata", strings.NewReader(body)) |
| 954 | + r.Header.Set("Content-Type", "application/json") |
| 955 | + signRequest(t, r, body) |
| 956 | + w := httptest.NewRecorder() |
| 957 | + tenant.ServeHTTP(w, r) |
| 958 | + if w.Code != http.StatusOK { |
| 959 | + t.Fatalf("metadata status=%d body=%s", w.Code, w.Body.String()) |
| 960 | + } |
| 961 | + var purpose, typ string |
| 962 | + var size int64 |
| 963 | + if err := tenant.store.DB().QueryRow("SELECT purpose,logical_size,logical_type FROM blob_claim_metadata WHERE sha256=? AND uploader=?", entry.SHA256, tenant.Policy().Owner).Scan(&purpose, &size, &typ); err != nil { |
| 964 | + t.Fatal(err) |
| 965 | + } |
| 966 | + if purpose != "file" || size != 104857600 || typ != "application/octet-stream" { |
| 967 | + t.Fatalf("metadata=%q %d %q", purpose, size, typ) |
| 968 | + } |
| 969 | + bad := strings.Replace(body, `"size":104857600`, `"size":104857601`, 1) |
| 970 | + r = httptest.NewRequest(http.MethodPost, "http://relay.test/files/metadata", strings.NewReader(bad)) |
| 971 | + r.Header.Set("Content-Type", "application/json") |
| 972 | + signRequest(t, r, body) |
| 973 | + w = httptest.NewRecorder() |
| 974 | + tenant.ServeHTTP(w, r) |
| 975 | + if w.Code != http.StatusUnauthorized { |
| 976 | + t.Fatalf("tampered metadata status=%d body=%s", w.Code, w.Body.String()) |
| 977 | + } |
| 978 | + unknown := body[:len(body)-1] + `,"key":"must-not-store"}` |
| 979 | + r = httptest.NewRequest(http.MethodPost, "http://relay.test/files/metadata", strings.NewReader(unknown)) |
| 980 | + r.Header.Set("Content-Type", "application/json") |
| 981 | + signRequest(t, r, unknown) |
| 982 | + w = httptest.NewRecorder() |
| 983 | + tenant.ServeHTTP(w, r) |
| 984 | + if w.Code != http.StatusBadRequest { |
| 985 | + t.Fatalf("unknown metadata status=%d body=%s", w.Code, w.Body.String()) |
| 986 | + } |
| 987 | +} |
| 988 | + |
| 989 | +func TestEncryptedUploadFinalizesOneLogicalLibraryFile(t *testing.T) { |
| 990 | + _, tenant := testTenant(t) |
| 991 | + body := "an encrypted upload fixture" |
| 992 | + r := httptest.NewRequest(http.MethodPut, "http://relay.test/upload?purpose=chunk", strings.NewReader(body)) |
| 993 | + r.Header.Set("Content-Type", "application/octet-stream") |
| 994 | + signRequest(t, r, body) |
| 995 | + w := httptest.NewRecorder() |
| 996 | + tenant.ServeHTTP(w, r) |
| 997 | + if w.Code != http.StatusOK { |
| 998 | + t.Fatalf("upload: %d %s", w.Code, w.Body.String()) |
| 999 | + } |
| 1000 | + var uploaded struct { |
| 1001 | + Hash string `json:"sha256"` |
| 1002 | + } |
| 1003 | + if err := json.Unmarshal(w.Body.Bytes(), &uploaded); err != nil { |
| 1004 | + t.Fatal(err) |
| 1005 | + } |
| 1006 | + query := []json.RawMessage{json.RawMessage(`{"view":"library"}`)} |
| 1007 | + library, err := tenant.Execute(t.Context(), tenant.Policy().Owner, "browsefiles", query) |
| 1008 | + if err != nil { |
| 1009 | + t.Fatal(err) |
| 1010 | + } |
| 1011 | + if len(fileItems(t, library)) != 0 { |
| 1012 | + t.Fatalf("internal chunk in library: %v", library) |
| 1013 | + } |
| 1014 | + metadata := `{"hash":"` + uploaded.Hash + `","name":"Encrypted file","purpose":"file","size":104857600,"type":"application/octet-stream"}` |
| 1015 | + r = httptest.NewRequest(http.MethodPost, "http://relay.test/files/metadata", strings.NewReader(metadata)) |
| 1016 | + signRequest(t, r, metadata) |
| 1017 | + w = httptest.NewRecorder() |
| 1018 | + tenant.ServeHTTP(w, r) |
| 1019 | + if w.Code != http.StatusOK { |
| 1020 | + t.Fatalf("finalize: %d %s", w.Code, w.Body.String()) |
| 1021 | + } |
| 1022 | + library, err = tenant.Execute(t.Context(), tenant.Policy().Owner, "browsefiles", query) |
| 1023 | + if err != nil { |
| 1024 | + t.Fatal(err) |
| 1025 | + } |
| 1026 | + items := fileItems(t, library) |
| 1027 | + if len(items) != 1 || items[0]["name"] != "Encrypted file" || items[0]["size"] != int64(104857600) { |
| 1028 | + t.Fatalf("logical file: %v", items) |
| 1029 | + } |
| 1030 | +} |
| 1031 | + |
| 1032 | +func TestFileMetadataCannotChangeAnotherUploadersClaim(t *testing.T) { |
| 1033 | + _, tenant := testTenant(t) |
| 1034 | + entry, err := tenant.blobs.Put(t.Context(), blob.PutOptions{Reader: strings.NewReader("someone else's file"), Uploader: strings.Repeat("b", 64)}) |
| 1035 | + if err != nil { |
| 1036 | + t.Fatal(err) |
| 1037 | + } |
| 1038 | + body := `{"hash":"` + entry.SHA256 + `","name":"Changed","purpose":"file"}` |
| 1039 | + r := httptest.NewRequest(http.MethodPost, "http://relay.test/files/metadata", strings.NewReader(body)) |
| 1040 | + signRequest(t, r, body) |
| 1041 | + w := httptest.NewRecorder() |
| 1042 | + tenant.ServeHTTP(w, r) |
| 1043 | + if w.Code == http.StatusOK { |
| 1044 | + t.Fatal("owner must not edit another uploader's claim metadata") |
| 1045 | + } |
| 1046 | + var count int |
| 1047 | + if err := tenant.store.DB().QueryRow("SELECT count(*) FROM blob_claim_metadata WHERE sha256=?", entry.SHA256).Scan(&count); err != nil { |
| 1048 | + t.Fatal(err) |
| 1049 | + } |
| 1050 | + if count != 0 { |
| 1051 | + t.Fatal("metadata changed without claim ownership") |
| 1052 | + } |
| 1053 | +} |
| 1054 | diff --git a/internal/daemon/mcp_attachments.go b/internal/daemon/mcp_attachments.go |
| 1055 | index a0c6a91..9dd6628 100644 |
| 1056 | --- a/internal/daemon/mcp_attachments.go |
| 1057 | +++ b/internal/daemon/mcp_attachments.go |
| 1058 | @@ -169,6 +169,9 @@ func (t *Tenant) mcpReadAttachment(ctx context.Context, call mcp.Call) (mcp.Resu |
| 1059 | if err := t.browseRead(ctx, call.Actor); err != nil { |
| 1060 | return mcp.Failure("attachment unavailable: "+err.Error(), nil), nil |
| 1061 | } |
| 1062 | + if err := t.fileReadAccess(ctx, call.Actor, call.String("sha256")); err != nil { |
| 1063 | + return mcp.Failure("attachment unavailable: "+err.Error(), nil), nil |
| 1064 | + } |
| 1065 | if err := t.roomAttachmentAccess(ctx, call.Actor, call.String("sha256")); err != nil { |
| 1066 | return mcp.Failure("attachment unavailable: "+err.Error(), nil), nil |
| 1067 | } |
| 1068 | diff --git a/internal/daemon/member_file_access_test.go b/internal/daemon/member_file_access_test.go |
| 1069 | new file mode 100644 |
| 1070 | index 0000000..d099702 |
| 1071 | --- /dev/null |
| 1072 | +++ b/internal/daemon/member_file_access_test.go |
| 1073 | @@ -0,0 +1,135 @@ |
| 1074 | +package daemon |
| 1075 | + |
| 1076 | +import ( |
| 1077 | + "context" |
| 1078 | + "encoding/json" |
| 1079 | + "net/http" |
| 1080 | + "net/http/httptest" |
| 1081 | + "strings" |
| 1082 | + "testing" |
| 1083 | +) |
| 1084 | + |
| 1085 | +func TestMemberFileAccessAcrossHTTPAndBrowseSurfaces(t *testing.T) { |
| 1086 | + h := newRoomHarness(t) |
| 1087 | + ctx := context.Background() |
| 1088 | + const body = "members-only file bytes" |
| 1089 | + request := httptest.NewRequest(http.MethodPut, "http://relay.test/upload?access=members&filename=private.txt", strings.NewReader(body)) |
| 1090 | + request.Header.Set("Content-Type", "text/plain") |
| 1091 | + signRequestWithSecret(t, request, body, h.secrets["alice"]) |
| 1092 | + uploaded := httptest.NewRecorder() |
| 1093 | + h.tenant.ServeHTTP(uploaded, request) |
| 1094 | + if uploaded.Code != http.StatusOK { |
| 1095 | + t.Fatalf("member upload = %d: %s", uploaded.Code, uploaded.Body.String()) |
| 1096 | + } |
| 1097 | + var descriptor struct { |
| 1098 | + Hash string `json:"sha256"` |
| 1099 | + } |
| 1100 | + if err := json.Unmarshal(uploaded.Body.Bytes(), &descriptor); err != nil || len(descriptor.Hash) != 64 { |
| 1101 | + t.Fatalf("upload descriptor = %s: %v", uploaded.Body.String(), err) |
| 1102 | + } |
| 1103 | + |
| 1104 | + if rows, err := h.browse("bob", "browsefiles", map[string]any{"view": "library", "limit": 50}); err != nil { |
| 1105 | + t.Fatalf("member library: %v", err) |
| 1106 | + } else if !libraryContainsHash(rows, descriptor.Hash, "private.txt") { |
| 1107 | + t.Fatalf("member library omitted private file: %v", rows) |
| 1108 | + } |
| 1109 | + if rows, err := h.browse("stranger", "browsefiles", map[string]any{"view": "library", "limit": 50}); err != nil { |
| 1110 | + t.Fatalf("outsider library: %v", err) |
| 1111 | + } else if libraryContainsHash(rows, descriptor.Hash, "private.txt") { |
| 1112 | + t.Fatalf("outsider library exposed private file: %v", rows) |
| 1113 | + } |
| 1114 | + |
| 1115 | + for _, route := range []string{"/" + descriptor.Hash, "/nip96/" + descriptor.Hash, "/files/raw?hash=" + descriptor.Hash, "/media/" + descriptor.Hash} { |
| 1116 | + for _, method := range []string{http.MethodGet, http.MethodHead} { |
| 1117 | + for _, actor := range []string{"owner", "bob"} { |
| 1118 | + t.Run(method+" "+actor+" "+route, func(t *testing.T) { |
| 1119 | + response := memberFileRequest(t, h, method, route, actor) |
| 1120 | + if response.Code != http.StatusOK { |
| 1121 | + t.Fatalf("%s response = %d: %s", actor, response.Code, response.Body.String()) |
| 1122 | + } |
| 1123 | + if method == http.MethodGet && response.Body.String() != body { |
| 1124 | + t.Fatalf("%s body = %q", actor, response.Body.String()) |
| 1125 | + } |
| 1126 | + }) |
| 1127 | + } |
| 1128 | + t.Run(method+" guest "+route, func(t *testing.T) { |
| 1129 | + response := memberFileRequest(t, h, method, route, "") |
| 1130 | + if response.Code != http.StatusUnauthorized && response.Code != http.StatusForbidden { |
| 1131 | + t.Fatalf("guest response = %d: %s", response.Code, response.Body.String()) |
| 1132 | + } |
| 1133 | + }) |
| 1134 | + t.Run(method+" outsider "+route, func(t *testing.T) { |
| 1135 | + response := memberFileRequest(t, h, method, route, "stranger") |
| 1136 | + if response.Code != http.StatusForbidden { |
| 1137 | + t.Fatalf("outsider response = %d: %s", response.Code, response.Body.String()) |
| 1138 | + } |
| 1139 | + }) |
| 1140 | + } |
| 1141 | + } |
| 1142 | + |
| 1143 | + for _, actor := range []string{"bob", "stranger"} { |
| 1144 | + request := httptest.NewRequest(http.MethodGet, "http://relay.test/file?hash="+descriptor.Hash, nil) |
| 1145 | + if actor != "" { |
| 1146 | + signRequestWithSecret(t, request, "", h.secrets[actor]) |
| 1147 | + } |
| 1148 | + response := httptest.NewRecorder() |
| 1149 | + h.tenant.ServeHTTP(response, request) |
| 1150 | + page := response.Body.String() |
| 1151 | + if actor == "bob" { |
| 1152 | + if response.Code != http.StatusOK || !strings.Contains(page, "private.txt") { |
| 1153 | + t.Fatalf("member file page = %d: %s", response.Code, page) |
| 1154 | + } |
| 1155 | + } else if strings.Contains(page, "private.txt") || strings.Contains(page, body) { |
| 1156 | + t.Fatalf("outsider file page leaked private data: %d: %s", response.Code, page) |
| 1157 | + } |
| 1158 | + } |
| 1159 | + |
| 1160 | + result, err := h.tenant.Execute(ctx, h.keys["bob"], "browsefile", []json.RawMessage{rawJSON(map[string]any{"hash": descriptor.Hash})}) |
| 1161 | + if err != nil || !strings.Contains(string(rawJSON(result)), "private.txt") { |
| 1162 | + t.Fatalf("member browsefile = %v, %v", result, err) |
| 1163 | + } |
| 1164 | + if _, err := h.tenant.Execute(ctx, h.keys["stranger"], "browsefile", []json.RawMessage{rawJSON(map[string]any{"hash": descriptor.Hash})}); err == nil { |
| 1165 | + t.Fatal("outsider browsefile exposed private file") |
| 1166 | + } |
| 1167 | + |
| 1168 | + if _, err := h.tenant.Execute(ctx, h.keys["owner"], "removemember", []json.RawMessage{rawJSON(h.keys["bob"])}); err != nil { |
| 1169 | + t.Fatalf("remove member: %v", err) |
| 1170 | + } |
| 1171 | + if response := memberFileRequest(t, h, http.MethodGet, "/files/raw?hash="+descriptor.Hash+"&check=revoked", "bob"); response.Code != http.StatusForbidden { |
| 1172 | + t.Fatalf("revoked member response = %d: %s", response.Code, response.Body.String()) |
| 1173 | + } |
| 1174 | + if _, err := h.tenant.Execute(ctx, h.keys["owner"], "setmember", []json.RawMessage{rawJSON(h.keys["bob"]), rawJSON(map[string]any{"role": "member"})}); err != nil { |
| 1175 | + t.Fatalf("restore member: %v", err) |
| 1176 | + } |
| 1177 | + if _, err := h.tenant.Execute(ctx, h.keys["owner"], "banpubkey", []json.RawMessage{rawJSON(h.keys["bob"]), rawJSON("test")}); err != nil { |
| 1178 | + t.Fatalf("ban member: %v", err) |
| 1179 | + } |
| 1180 | + if response := memberFileRequest(t, h, http.MethodGet, "/files/raw?hash="+descriptor.Hash+"&check=banned", "bob"); response.Code != http.StatusForbidden { |
| 1181 | + t.Fatalf("banned member response = %d: %s", response.Code, response.Body.String()) |
| 1182 | + } |
| 1183 | +} |
| 1184 | + |
| 1185 | +func memberFileRequest(t *testing.T, h *roomHarness, method, route, actor string) *httptest.ResponseRecorder { |
| 1186 | + t.Helper() |
| 1187 | + request := httptest.NewRequest(method, "http://relay.test"+route, nil) |
| 1188 | + if actor != "" { |
| 1189 | + signRequestWithSecret(t, request, "", h.secrets[actor]) |
| 1190 | + } |
| 1191 | + response := httptest.NewRecorder() |
| 1192 | + h.tenant.ServeHTTP(response, request) |
| 1193 | + return response |
| 1194 | +} |
| 1195 | + |
| 1196 | +func libraryContainsHash(rows map[string]any, hash, name string) bool { |
| 1197 | + items, ok := rows["items"].([]any) |
| 1198 | + if !ok { |
| 1199 | + return false |
| 1200 | + } |
| 1201 | + for _, raw := range items { |
| 1202 | + item, ok := raw.(map[string]any) |
| 1203 | + if ok && item["sha256"] == hash && item["name"] == name { |
| 1204 | + return true |
| 1205 | + } |
| 1206 | + } |
| 1207 | + return false |
| 1208 | +} |
| 1209 | diff --git a/internal/daemon/services.go b/internal/daemon/services.go |
| 1210 | index 3d80178..261bcf7 100644 |
| 1211 | --- a/internal/daemon/services.go |
| 1212 | +++ b/internal/daemon/services.go |
| 1213 | @@ -69,6 +69,13 @@ func (t *Tenant) initServices(ctx context.Context) error { |
| 1214 | return t.community.SubmitReportTx(ctx, tx, reporter, target, targetType, reportType, content, t.Policy().ReportThreshold) |
| 1215 | }, |
| 1216 | CanRead: func(ctx context.Context, hash string, pubkeys []string) bool { |
| 1217 | + access, accessErr := t.blobs.BlobAccess(ctx, hash) |
| 1218 | + if accessErr != nil { |
| 1219 | + return false |
| 1220 | + } |
| 1221 | + if access == blob.AccessMembers && len(pubkeys) == 0 { |
| 1222 | + return false |
| 1223 | + } |
| 1224 | keys := []string{} |
| 1225 | for _, pubkey := range pubkeys { |
| 1226 | if pubkey == "" { |
| 1227 | @@ -78,8 +85,17 @@ func (t *Tenant) initServices(ctx context.Context) error { |
| 1228 | if err != nil || banned { |
| 1229 | return false |
| 1230 | } |
| 1231 | + if access == blob.AccessMembers { |
| 1232 | + role, roleErr := t.community.Role(ctx, pubkey) |
| 1233 | + if roleErr != nil || !t.memberBlobReader(ctx, pubkey, role) { |
| 1234 | + return false |
| 1235 | + } |
| 1236 | + } |
| 1237 | keys = append(keys, pubkey) |
| 1238 | } |
| 1239 | + if access == blob.AccessMembers && len(keys) == 0 { |
| 1240 | + return false |
| 1241 | + } |
| 1242 | _, err := t.gate.Read(ctx, []event.Filter{{}}, relay.Session{PubKeys: keys, RelayURL: t.RelayURL()}) |
| 1243 | return err == nil |
| 1244 | }, IsOwner: func(pubkey string) bool { return pubkey == t.Policy().Owner }, |
| 1245 | @@ -94,6 +110,11 @@ func (t *Tenant) initServices(ctx context.Context) error { |
| 1246 | t.sites, err = sites.New(sites.Config{ |
| 1247 | Store: t.store, |
| 1248 | GetBlob: func(ctx context.Context, hash string) (sites.Blob, error) { |
| 1249 | + if access, accessErr := t.blobs.BlobAccess(ctx, hash); accessErr != nil { |
| 1250 | + return sites.Blob{}, accessErr |
| 1251 | + } else if access == blob.AccessMembers { |
| 1252 | + return sites.Blob{}, errors.New("restricted: member files cannot be published as sites") |
| 1253 | + } |
| 1254 | rooms, err := t.roomAttachmentRooms(ctx, hash) |
| 1255 | if err != nil { |
| 1256 | return sites.Blob{}, err |
| 1257 | @@ -172,6 +193,45 @@ func (t *Tenant) initServices(ctx context.Context) error { |
| 1258 | return nil |
| 1259 | } |
| 1260 | |
| 1261 | +func (t *Tenant) memberBlobReader(ctx context.Context, pubkey, role string) bool { |
| 1262 | + if role == "owner" || role == "moderator" || role == "member" { |
| 1263 | + return true |
| 1264 | + } |
| 1265 | + if role != "agent" { |
| 1266 | + return false |
| 1267 | + } |
| 1268 | + _, err := t.agentUploadGrant(ctx, pubkey) |
| 1269 | + return err == nil |
| 1270 | +} |
| 1271 | + |
| 1272 | +func (t *Tenant) fileReadAccess(ctx context.Context, actor, hash string) error { |
| 1273 | + access, err := t.blobs.BlobAccess(ctx, hash) |
| 1274 | + if err != nil { |
| 1275 | + return err |
| 1276 | + } |
| 1277 | + if access != blob.AccessMembers { |
| 1278 | + return nil |
| 1279 | + } |
| 1280 | + if actor == "" { |
| 1281 | + return errors.New("auth-required: member file requires sign in") |
| 1282 | + } |
| 1283 | + banned, err := t.community.IsBanned(ctx, actor) |
| 1284 | + if err != nil { |
| 1285 | + return fmt.Errorf("check file reader ban: %w", err) |
| 1286 | + } |
| 1287 | + if banned { |
| 1288 | + return errors.New("blocked: file reader is banned") |
| 1289 | + } |
| 1290 | + role, err := t.community.Role(ctx, actor) |
| 1291 | + if err != nil { |
| 1292 | + return fmt.Errorf("check file reader role: %w", err) |
| 1293 | + } |
| 1294 | + if !t.memberBlobReader(ctx, actor, role) { |
| 1295 | + return errors.New("restricted: relay membership required to read this file") |
| 1296 | + } |
| 1297 | + return nil |
| 1298 | +} |
| 1299 | + |
| 1300 | // backend adapts the management method signature to webui's form-oriented API. |
| 1301 | type backend struct{ tenant *Tenant } |
| 1302 | |
| 1303 | @@ -440,14 +500,21 @@ func (t *Tenant) validateBlobUpload(r *http.Request, hash string) error { |
| 1304 | // signed request can be verified before parsing or fetching any content. |
| 1305 | func blobAuthBody(r *http.Request, action blob.Action) (string, error) { |
| 1306 | buffer := action == blob.ActionMirror || action == blob.ActionReport || |
| 1307 | - (action == blob.ActionUpload && r.Method == http.MethodPost && r.URL.Path == "/nip96") |
| 1308 | + (action == blob.ActionUpload && r.Method == http.MethodPost && (r.URL.Path == "/nip96" || r.URL.Path == "/files/metadata")) |
| 1309 | if !buffer { |
| 1310 | return "", nil |
| 1311 | } |
| 1312 | - body, err := io.ReadAll(r.Body) |
| 1313 | + var source io.Reader = r.Body |
| 1314 | + if r.URL.Path == "/files/metadata" { |
| 1315 | + source = io.LimitReader(r.Body, 16*1024+1) |
| 1316 | + } |
| 1317 | + body, err := io.ReadAll(source) |
| 1318 | if err != nil { |
| 1319 | return "", fmt.Errorf("read blob authorization body: %w", err) |
| 1320 | } |
| 1321 | + if r.URL.Path == "/files/metadata" && len(body) > 16*1024 { |
| 1322 | + return "", errors.New("invalid: file metadata exceeds 16 KiB") |
| 1323 | + } |
| 1324 | r.Body = io.NopCloser(bytes.NewReader(body)) |
| 1325 | return string(body), nil |
| 1326 | } |
| 1327 | diff --git a/internal/webui/blossom-upload.js b/internal/webui/blossom-upload.js |
| 1328 | index 20d5602..382dd70 100644 |
| 1329 | --- a/internal/webui/blossom-upload.js |
| 1330 | +++ b/internal/webui/blossom-upload.js |
| 1331 | @@ -6,6 +6,8 @@ |
| 1332 | // executes that plan with retry, progress and cancellation support. |
| 1333 | |
| 1334 | const {sha256, hex} = globalThis.tiny.util; |
| 1335 | + const probeCache = new Map(); |
| 1336 | + const DEFAULT_PROBE_CACHE_MS = 5 * 60 * 1000; |
| 1337 | const asBytes = async value => { |
| 1338 | if (value instanceof Uint8Array) return value; |
| 1339 | if (ArrayBuffer.isView(value)) return new Uint8Array(value.buffer, value.byteOffset, value.byteLength); |
| 1340 | @@ -30,6 +32,17 @@ |
| 1341 | throw new Error("invalid Blossom upload descriptor"); |
| 1342 | return descriptor; |
| 1343 | }; |
| 1344 | + const withMetadata = (target, metadata) => { |
| 1345 | + if (metadata === undefined) return target; |
| 1346 | + if (!metadata || typeof metadata !== "object" || Array.isArray(metadata)) throw new Error("upload metadata must be a map"); |
| 1347 | + const parsed = new URL(target, globalThis.location?.href || "https://blossom.invalid"); |
| 1348 | + for (const [key, value] of Object.entries(metadata)) { |
| 1349 | + if (!/^[A-Za-z0-9_-]{1,64}$/.test(key) || value === undefined || typeof value === "object") |
| 1350 | + throw new Error("invalid upload metadata"); |
| 1351 | + parsed.searchParams.set(key, String(value)); |
| 1352 | + } |
| 1353 | + return parsed.href; |
| 1354 | + }; |
| 1355 | |
| 1356 | // resolveTarget checks the source and options and returns the blob URL |
| 1357 | // the upload addresses. Both the live uploader and the background plan |
| 1358 | @@ -58,7 +71,7 @@ |
| 1359 | const target = new RegExp("/" + hash + "(?:\\.[a-z0-9]{1,8})?/?$").test(path) |
| 1360 | ? base |
| 1361 | : base.replace(/\/$/, "") + "/" + hash + (extension ? "." + extension : ""); |
| 1362 | - return {bytes, hash, type, url: target}; |
| 1363 | + return {bytes, hash, type, url: withMetadata(target, options.metadata), probeURL: base.replace(/\/$/, "")}; |
| 1364 | }; |
| 1365 | |
| 1366 | // plan lists the signed chunk requests for one upload without sending |
| 1367 | @@ -89,13 +102,15 @@ |
| 1368 | }; |
| 1369 | |
| 1370 | const uploadTask = async (source, options = {}, controller) => { |
| 1371 | - const {bytes, hash, type, url} = await resolveTarget(source, options); |
| 1372 | + const {bytes, hash, type, url, probeURL} = await resolveTarget(source, options); |
| 1373 | const fetcher = options.fetch || globalThis.fetch; |
| 1374 | if (typeof fetcher !== "function") throw new Error("fetch is unavailable"); |
| 1375 | const chunkSize = |
| 1376 | Number.isSafeInteger(options.chunkSize) && options.chunkSize > 0 ? options.chunkSize : 5 * 1024 * 1024; |
| 1377 | const timeoutMs = Number.isSafeInteger(options.timeoutMs) && options.timeoutMs > 0 ? options.timeoutMs : 30000; |
| 1378 | - const retries = Number.isSafeInteger(options.retries) && options.retries >= 0 ? options.retries : 0; |
| 1379 | + const retries = Number.isSafeInteger(options.retries) && options.retries >= 0 ? options.retries : 2; |
| 1380 | + const retryDelayMs = Number.isFinite(options.retryDelayMs) && options.retryDelayMs >= 0 ? options.retryDelayMs : 250; |
| 1381 | + const retryMaxDelayMs = Number.isFinite(options.retryMaxDelayMs) && options.retryMaxDelayMs >= 0 ? options.retryMaxDelayMs : 5000; |
| 1382 | const state = options.state instanceof Map ? options.state : new Map(); |
| 1383 | const notify = (offset, length, status) => options.onState?.({offset, length, status, state}); |
| 1384 | const progress = sent => |
| 1385 | @@ -159,6 +174,23 @@ |
| 1386 | requestController.abort(); |
| 1387 | } |
| 1388 | }; |
| 1389 | + const waitForRetry = async attempt => { |
| 1390 | + const delay = Math.min(retryMaxDelayMs, retryDelayMs * 2 ** Math.max(0, attempt - 1)); |
| 1391 | + if (!delay) return; |
| 1392 | + await new Promise((resolve, reject) => { |
| 1393 | + let timer; |
| 1394 | + const cancel = () => { |
| 1395 | + clearTimeout(timer); |
| 1396 | + controller.signal.removeEventListener("abort", cancel); |
| 1397 | + reject(abortError()); |
| 1398 | + }; |
| 1399 | + timer = setTimeout(() => { |
| 1400 | + controller.signal.removeEventListener("abort", cancel); |
| 1401 | + resolve(); |
| 1402 | + }, delay); |
| 1403 | + controller.signal.addEventListener("abort", cancel, {once: true}); |
| 1404 | + }); |
| 1405 | + }; |
| 1406 | const retry = async (action, offset, length) => { |
| 1407 | let attempt = 0; |
| 1408 | while (true) { |
| 1409 | @@ -167,6 +199,7 @@ |
| 1410 | const result = await action(); |
| 1411 | if (!result.ok && result.status >= 500 && attempt++ < retries) { |
| 1412 | notify(offset, length, "retrying"); |
| 1413 | + await waitForRetry(attempt); |
| 1414 | continue; |
| 1415 | } |
| 1416 | return result; |
| 1417 | @@ -174,15 +207,21 @@ |
| 1418 | if (controller.signal.aborted) throw abortError(); |
| 1419 | if (attempt++ >= retries) throw error; |
| 1420 | notify(offset, length, "retrying"); |
| 1421 | + await waitForRetry(attempt); |
| 1422 | } |
| 1423 | } |
| 1424 | }; |
| 1425 | |
| 1426 | - let supportsPatch = options.probe === false; |
| 1427 | - if (options.probe !== false) { |
| 1428 | + const cacheMs = options.probeCache === false ? 0 : |
| 1429 | + Number.isFinite(options.probeCacheMs) && options.probeCacheMs >= 0 ? options.probeCacheMs : DEFAULT_PROBE_CACHE_MS; |
| 1430 | + const cachedProbe = cacheMs ? probeCache.get(probeURL) : null; |
| 1431 | + const cacheValid = Boolean(cachedProbe && cachedProbe.expiresAt > Date.now()); |
| 1432 | + let supportsPatch = options.probe === false || (cacheValid && cachedProbe.supportsPatch); |
| 1433 | + if (options.probe !== false && !cacheValid) { |
| 1434 | try { |
| 1435 | const response = await request("OPTIONS", null, {}); |
| 1436 | supportsPatch = /(?:^|,|\s)PATCH(?:,|\s|$)/i.test(response.headers?.get?.("allow") || ""); |
| 1437 | + if (cacheMs) probeCache.set(probeURL, {supportsPatch, expiresAt: Date.now() + cacheMs}); |
| 1438 | } catch { |
| 1439 | supportsPatch = false; |
| 1440 | } |
| 1441 | diff --git a/internal/webui/browse.html b/internal/webui/browse.html |
| 1442 | index 36b2781..71d3c77 100644 |
| 1443 | --- a/internal/webui/browse.html |
| 1444 | +++ b/internal/webui/browse.html |
| 1445 | @@ -12,17 +12,17 @@ |
| 1446 | {{end}}</section> |
| 1447 | {{end}} |
| 1448 | |
| 1449 | -{{define "files"}}{{$data := asMap .Event}}{{$view := or (str (index $data "view")) (.Query.Get "view") "library"}}<section id="file-library"> |
| 1450 | +{{define "files"}}{{$data := asMap .Event}}{{$view := or (str (index $data "view")) (.Query.Get "view") "library"}}<section id="file-library" data-pubkey="{{.Actor}}"> |
| 1451 | <h1>Files</h1> |
| 1452 | <nav id="file-views" aria-label="File collections"><a href="/files"{{if eq $view "library"}} aria-current="page"{{end}}>My files</a><a href="/files?view=sites"{{if eq $view "sites"}} aria-current="page"{{end}}>Sites</a><a href="/files?view=rooms"{{if eq $view "rooms"}} aria-current="page"{{end}}>Rooms</a>{{if index $data "can_manage"}}<a href="/files?view=storage"{{if eq $view "storage"}} aria-current="page"{{end}}>Storage</a>{{end}}</nav> |
| 1453 | -<p id="file-context">{{if eq $view "sites"}}Published assets, organized by site.{{else if eq $view "rooms"}}Attachments from rooms you can access.{{else if eq $view "storage"}}All stored files, newest first.{{else}}Your uploads and folders. Site assets and room attachments have their own collections.{{end}}</p> |
| 1454 | +<p id="file-context">{{if eq $view "sites"}}Published assets, organized by site.{{else if eq $view "rooms"}}Attachments from rooms you can access.{{else if eq $view "storage"}}All stored files, newest first.{{else}}Your uploads, folders and files shared with relay members.{{end}}</p> |
| 1455 | <form method="get" action="/files" id="file-search"><input type="hidden" name="view" value="{{$view}}">{{with .Query.Get "path"}}<input type="hidden" name="path" value="{{.}}">{{end}}<label>Search <input name="q" value="{{.Query.Get "q"}}" placeholder="Name, type or hash" autocomplete="off"></label> <button>Search</button>{{if .Query.Get "q"}} <a href="{{browsePageURL "/files" .Query "q" ""}}">Clear</a>{{end}}</form> |
| 1456 | {{with index $data "breadcrumbs"}}<nav id="file-breadcrumbs" aria-label="Folder path">{{range $i, $raw := .}}{{$crumb := asMap $raw}}{{if $i}} / {{end}}<a href="{{str (index $crumb "open")}}">{{str (index $crumb "name")}}</a>{{end}}</nav>{{end}} |
| 1457 | {{if .Error}}{{template "alert" .Error}} |
| 1458 | {{else if .Feed}}<table id="rows"> |
| 1459 | <thead><tr><th scope="col">Name</th><th scope="col">Details</th><th scope="col">Added</th></tr></thead> |
| 1460 | <tbody> |
| 1461 | -{{range .Feed}}{{$f := asMap .}}{{$kind := str (index $f "kind")}}{{$hash := str (index $f "sha256")}}{{$name := or (str (index $f "name")) (shortID $hash)}}{{$open := or (str (index $f "open")) (printf "/file?hash=%s" (urlquery $hash))}}<tr data-kind="{{$kind}}"><td><a href="{{$open}}">{{if and $hash (hasPrefix (str (index $f "type")) "image/")}}<img loading="lazy" src="/files/raw?hash={{urlquery $hash}}" alt="">{{else}}<span data-file-icon aria-hidden="true">{{if or (eq $kind "folder") (eq $kind "site") (eq $kind "room")}}â–±{{else}}â–¤{{end}}</span>{{end}}<span>{{$name}}</span></a>{{with index $f "context_url"}}<small><a href="{{.}}">{{str (index $f "context")}}</a></small>{{end}}</td><td>{{if eq $kind "folder"}}Folder{{else if eq $kind "site"}}Site{{else if eq $kind "room"}}Room{{else}}{{fileSize (index $f "size")}}<small>{{str (index $f "type")}}</small>{{end}}</td><td>{{with index $f "uploaded"}}<time datetime="{{datetime .}}">{{when .}}</time>{{end}}</td></tr> |
| 1462 | +{{range .Feed}}{{$f := asMap .}}{{$kind := str (index $f "kind")}}{{$hash := str (index $f "sha256")}}{{$name := or (str (index $f "name")) (shortID $hash)}}{{$open := or (str (index $f "open")) (printf "/file?hash=%s" (urlquery $hash))}}<tr data-kind="{{$kind}}" data-hash="{{$hash}}"><td><a href="{{$open}}">{{if and $hash (hasPrefix (str (index $f "type")) "image/")}}<img loading="lazy" src="/files/raw?hash={{urlquery $hash}}" alt="">{{else}}<span data-file-icon aria-hidden="true">{{if or (eq $kind "folder") (eq $kind "site") (eq $kind "room")}}â–±{{else}}â–¤{{end}}</span>{{end}}<span>{{$name}}</span></a>{{with index $f "context_url"}}<small><a href="{{.}}">{{str (index $f "context")}}</a></small>{{end}}</td><td>{{if eq $kind "folder"}}Folder{{else if eq $kind "site"}}Site{{else if eq $kind "room"}}Room{{else}}{{fileSize (index $f "size")}}<small>{{str (index $f "type")}}</small>{{if eq (str (index $f "access")) "members"}}<small>Relay members</small>{{else if eq (str (index $f "purpose")) "file"}}<small>Secret link</small>{{end}}{{end}}</td><td>{{with index $f "uploaded"}}<time datetime="{{datetime .}}">{{when .}}</time>{{end}}</td></tr> |
| 1463 | {{end}}</tbody> |
| 1464 | </table> |
| 1465 | {{else}}<p id="file-empty">{{if .Query.Get "q"}}No files match this search.{{else if eq $view "sites"}}No site files here yet.{{else if eq $view "rooms"}}No room attachments here yet.{{else if .Query.Get "path"}}This folder is empty.{{else}}No files here yet. Use Upload to add files or a folder.{{end}}</p> |
| 1466 | @@ -31,18 +31,12 @@ |
| 1467 | {{end}} |
| 1468 | |
| 1469 | {{define "file"}}{{if .Error}}<section><h1>File</h1>{{template "alert" .Error}}</section> |
| 1470 | -{{else}}{{$file := asMap .Event}}{{$hash := or (.Query.Get "hash") (.Query.Get "sha")}}<section id="file-view"> |
| 1471 | -<h1>{{or (str (index $file "name")) "File"}}</h1> |
| 1472 | -<table id="facts"> |
| 1473 | -<tr><th>sha256</th><td><code>{{str (index $file "sha256")}}</code></td></tr> |
| 1474 | -<tr><th>type</th><td>{{str (index $file "type")}}</td></tr> |
| 1475 | -<tr><th>size</th><td>{{str (index $file "size")}} bytes</td></tr> |
| 1476 | -{{with index $file "blossom_uri"}}<tr><th>blossom</th><td><code>{{.}}</code></td></tr>{{end}} |
| 1477 | -<tr><th>download</th><td><a fx-ignore href="/files/raw?hash={{urlquery (str (index $file "sha256"))}}">raw</a></td></tr> |
| 1478 | -</table> |
| 1479 | +{{else}}{{$file := asMap .Event}}{{$hash := or (.Query.Get "hash") (.Query.Get "sha")}}<section id="file-view" data-pubkey="{{.Actor}}"> |
| 1480 | +<h1>{{or (str (index $file "name")) "File"}}</h1>{{if eq (str (index $file "access")) "members"}}<p>Relay members can view and download this file.</p>{{end}} |
| 1481 | + |
| 1482 | <file-workspace-root hash="{{str (index $file "sha256")}}" type="{{str (index $file "type")}}" hidden></file-workspace-root> |
| 1483 | <noscript><p>Enable JavaScript to decrypt files and browse folder manifests locally. Raw downloads remain available above.</p></noscript> |
| 1484 | -<file-tools hash="{{str (index $file "sha256")}}" type="{{str (index $file "type")}}" size="{{str (index $file "size")}}" name="{{or (str (index $file "name")) (shortID (index $file "sha256"))}}"></file-tools> |
| 1485 | + |
| 1486 | {{$type := str (index $file "type")}}{{$raw := printf "/files/raw?hash=%s" (urlquery (str (index $file "sha256")))}}{{if hasPrefix $type "image/"}}<div id="media"><img src="{{$raw}}" alt=""></div> |
| 1487 | {{else if hasPrefix $type "video/"}}<div id="media"><video controls src="{{$raw}}"></video></div> |
| 1488 | {{else if hasPrefix $type "audio/"}}<div id="media"><audio controls src="{{$raw}}"></audio></div> |
| 1489 | @@ -50,7 +44,13 @@ |
| 1490 | {{else if index $file "binary"}}<p id="media">This is a binary file. Download it to view its contents.</p> |
| 1491 | {{else if index $file "content"}}{{sourceHTML .Event}} |
| 1492 | {{else}}<p id="media">This file is empty.</p> |
| 1493 | -{{end}}{{if .Actor}}<details id="manage-file"><summary>Manage file</summary><rpc-form method="deleteblob"><input type="hidden" name="param" value="{{$hash}}"><button>Delete file</button></rpc-form></details>{{end}}</section> |
| 1494 | +{{end}}<file-tools hash="{{str (index $file "sha256")}}" type="{{str (index $file "type")}}" size="{{str (index $file "size")}}" name="{{or (str (index $file "name")) (shortID (index $file "sha256"))}}"></file-tools><details id="file-facts"><summary>File details</summary><table id="facts"> |
| 1495 | +<tr><th>sha256</th><td><code>{{str (index $file "sha256")}}</code></td></tr> |
| 1496 | +<tr><th>type</th><td>{{str (index $file "type")}}</td></tr> |
| 1497 | +<tr><th>size</th><td>{{fileSize (index $file "size")}}</td></tr> |
| 1498 | +{{with index $file "blossom_uri"}}<tr><th>blossom</th><td><code>{{.}}</code></td></tr>{{end}} |
| 1499 | +<tr><th>download</th><td><a fx-ignore href="/files/raw?hash={{urlquery (str (index $file "sha256"))}}">raw</a></td></tr> |
| 1500 | +</table></details>{{if .Actor}}<details id="manage-file"><summary>Manage file</summary><rpc-form method="deleteblob"><input type="hidden" name="param" value="{{$hash}}"><button>Delete file</button></rpc-form></details>{{end}}</section> |
| 1501 | {{end}}{{end}} |
| 1502 | |
| 1503 | {{define "approvals"}}<section> |
| 1504 | diff --git a/internal/webui/components.js b/internal/webui/components.js |
| 1505 | index 5a5aad1..7ee426d 100644 |
| 1506 | --- a/internal/webui/components.js |
| 1507 | +++ b/internal/webui/components.js |
| 1508 | @@ -298,6 +298,7 @@ |
| 1509 | class FileTools extends HTMLElement { |
| 1510 | disconnectedCallback() { |
| 1511 | if (this.downloadURL) URL.revokeObjectURL(this.downloadURL); |
| 1512 | + if (this.previewURL && this.previewURL !== this.downloadURL) URL.revokeObjectURL(this.previewURL); |
| 1513 | } |
| 1514 | |
| 1515 | connectedCallback() { |
| 1516 | @@ -359,10 +360,52 @@ |
| 1517 | } |
| 1518 | if (params.get("ox") && await tiny.sha256hex(plain) !== params.get("ox")) throw Error("plaintext hash mismatch"); |
| 1519 | if (this.downloadURL) URL.revokeObjectURL(this.downloadURL); |
| 1520 | - const link = el("a"); this.downloadURL = URL.createObjectURL(new Blob([plain], {type: params.get("type") || this.type()})); link.href = this.downloadURL; link.download = params.get("name") || "decrypted-file"; link.textContent = "download decrypted file"; this.append(link); this.say("Decrypted locally."); |
| 1521 | + if (this.previewURL && this.previewURL !== this.downloadURL) URL.revokeObjectURL(this.previewURL); |
| 1522 | + const contentType = normalizePreviewType(params.get("type") || this.type()); |
| 1523 | + const type = safeMediaPreviewType(contentType); |
| 1524 | + const name = params.get("name") || "decrypted-file"; |
| 1525 | + this.previewURL = URL.createObjectURL(new Blob([plain], {type: type || "application/octet-stream"})); |
| 1526 | + this.renderPreview(contentType, name, plain); |
| 1527 | + const link = el("a"); this.downloadURL = this.previewURL; link.href = this.downloadURL; link.download = name; link.textContent = "download decrypted file"; this.append(link); this.say("Decrypted locally."); |
| 1528 | } catch (error) { this.say("Could not decrypt this file: " + error.message, true); } |
| 1529 | } |
| 1530 | |
| 1531 | + renderPreview(type, name, bytes) { |
| 1532 | + this.querySelector("[data-decrypted-preview]")?.remove(); |
| 1533 | + const text = safeTextPreviewType(type); |
| 1534 | + if (!this.previewURL || (!safeMediaPreviewType(type) && !text)) return; |
| 1535 | + const wrapper = el("div"); |
| 1536 | + wrapper.setAttribute("data-decrypted-preview", ""); |
| 1537 | + const heading = el("p", "Decrypted preview: " + name); |
| 1538 | + heading.setAttribute("data-preview-name", ""); |
| 1539 | + let media; |
| 1540 | + if (text) { |
| 1541 | + const pre = el("pre"); |
| 1542 | + const limit = 256 * 1024; |
| 1543 | + const truncated = bytes.byteLength > limit; |
| 1544 | + pre.textContent = new TextDecoder().decode(bytes.slice(0, limit)) + (truncated ? "\n\n[Preview truncated. Download the file to see the rest.]" : ""); |
| 1545 | + wrapper.append(heading, pre); |
| 1546 | + this.append(wrapper); |
| 1547 | + return; |
| 1548 | + } |
| 1549 | + if (type.startsWith("image/")) { |
| 1550 | + media = el("img"); |
| 1551 | + media.alt = name; |
| 1552 | + } else if (type.startsWith("video/")) { |
| 1553 | + media = el("video"); |
| 1554 | + media.controls = true; |
| 1555 | + media.preload = "metadata"; |
| 1556 | + } else if (type.startsWith("audio/")) { |
| 1557 | + media = el("audio"); |
| 1558 | + media.controls = true; |
| 1559 | + media.preload = "metadata"; |
| 1560 | + } |
| 1561 | + if (!media) return; |
| 1562 | + media.src = this.previewURL; |
| 1563 | + wrapper.append(heading, media); |
| 1564 | + this.append(wrapper); |
| 1565 | + } |
| 1566 | + |
| 1567 | async share(form) { |
| 1568 | if (this.sharing) return; |
| 1569 | await this.ready; |
| 1570 | @@ -384,6 +427,22 @@ |
| 1571 | } |
| 1572 | } |
| 1573 | |
| 1574 | + const normalizePreviewType = value => String(value || "").toLowerCase().split(";", 1)[0].trim(); |
| 1575 | + const safeMediaPreviewType = value => { |
| 1576 | + const type = normalizePreviewType(value); |
| 1577 | + if (/^image\/(?:avif|gif|jpeg|png|webp)$/.test(type)) return type; |
| 1578 | + if (/^video\/(?:mp4|ogg|webm)$/.test(type)) return type; |
| 1579 | + if (/^audio\/(?:aac|flac|mpeg|mp4|ogg|wav|webm)$/.test(type)) return type; |
| 1580 | + return ""; |
| 1581 | + }; |
| 1582 | + const safeTextPreviewType = value => { |
| 1583 | + const type = normalizePreviewType(value); |
| 1584 | + if (/^text\/(?:plain|markdown|css|javascript|x-(?:c|csrc|c\+\+src|java-source))$/.test(type)) return type; |
| 1585 | + if (/^application\/(?:json|javascript|xml|x-(?:javascript|yaml))$/.test(type)) return type; |
| 1586 | + if (type === "image/svg+xml" || type === "text/html" || type === "application/xhtml+xml") return type; |
| 1587 | + return ""; |
| 1588 | + }; |
| 1589 | + |
| 1590 | class PublishList extends FormElement { |
| 1591 | async submit(form) { |
| 1592 | const tag = this.getAttribute("tag"); |
| 1593 | diff --git a/internal/webui/file-catalog.js b/internal/webui/file-catalog.js |
| 1594 | new file mode 100644 |
| 1595 | index 0000000..c5e42c3 |
| 1596 | --- /dev/null |
| 1597 | +++ b/internal/webui/file-catalog.js |
| 1598 | @@ -0,0 +1,164 @@ |
| 1599 | +// Browser-local names and keys for encrypted files. Nothing in this catalog |
| 1600 | +// is sent to the relay; the tenant and signed-in key are part of its scope. |
| 1601 | +(() => { |
| 1602 | + "use strict"; |
| 1603 | + |
| 1604 | + const MAX_ENTRIES = 1000; |
| 1605 | + const MAX_NAME = 1024; |
| 1606 | + const HASH = /^[0-9a-f]{64}$/; |
| 1607 | + const PUBKEY = /^[0-9a-f]{64}$/; |
| 1608 | + |
| 1609 | + const actorKey = value => { |
| 1610 | + const key = String(value || "").trim().toLowerCase(); |
| 1611 | + return PUBKEY.test(key) ? key : ""; |
| 1612 | + }; |
| 1613 | + const tenantPath = () => { |
| 1614 | + try { |
| 1615 | + const local = globalThis.tiny?.localPath; |
| 1616 | + const value = new URL(typeof local === "function" ? local("/") : "/", location.href); |
| 1617 | + return value.pathname.replace(/\/+$/, "") || "/"; |
| 1618 | + } catch { |
| 1619 | + return "/"; |
| 1620 | + } |
| 1621 | + }; |
| 1622 | + const storageKey = pubkey => `tiny.files.catalog.v1:${location.origin}:${tenantPath()}:${pubkey}`; |
| 1623 | + const storage = () => { |
| 1624 | + try { return globalThis.localStorage; } catch { return null; } |
| 1625 | + }; |
| 1626 | + const read = pubkey => { |
| 1627 | + const key = actorKey(pubkey), store = storage(); |
| 1628 | + if (!key || !store) return []; |
| 1629 | + try { |
| 1630 | + const value = JSON.parse(store.getItem(storageKey(key)) || "[]"); |
| 1631 | + return Array.isArray(value) ? value.filter(entry => validEntry(entry, key)) : []; |
| 1632 | + } catch { |
| 1633 | + return []; |
| 1634 | + } |
| 1635 | + }; |
| 1636 | + const validName = value => |
| 1637 | + typeof value === "string" && value.length > 0 && value.length <= MAX_NAME && |
| 1638 | + value.isWellFormed?.() !== false && !/[\u0000/\\]/.test(value); |
| 1639 | + const validPath = value => value === undefined || |
| 1640 | + (typeof value === "string" && value.length <= MAX_NAME && !/[\u0000\\]/.test(value)); |
| 1641 | + const validLink = (value, hash) => { |
| 1642 | + if (typeof value !== "string") return false; |
| 1643 | + try { |
| 1644 | + const url = new URL(value, location.href); |
| 1645 | + const expected = new URL(globalThis.tiny?.localPath?.("/file") || "/file", location.href); |
| 1646 | + if (url.origin !== location.origin || url.pathname !== expected.pathname) return false; |
| 1647 | + if (url.searchParams.get("hash") !== hash && url.searchParams.get("sha") !== hash) return false; |
| 1648 | + const params = new URLSearchParams(url.hash.slice(1)); |
| 1649 | + return Boolean((params.get("key") && params.get("iv")) || |
| 1650 | + (params.get("enc") === "chk-v1" && params.get("key")) || |
| 1651 | + (params.get("manifest") === hash && params.get("key"))); |
| 1652 | + } catch { |
| 1653 | + return false; |
| 1654 | + } |
| 1655 | + }; |
| 1656 | + const validEntry = (entry, pubkey) => Boolean(entry && typeof entry === "object" && |
| 1657 | + actorKey(pubkey) && HASH.test(entry.hash) && validLink(entry.link, entry.hash) && |
| 1658 | + validName(entry.name) && (entry.type === undefined || typeof entry.type === "string") && |
| 1659 | + (entry.size === undefined || (Number.isSafeInteger(entry.size) && entry.size >= 0)) && |
| 1660 | + (entry.folder === undefined || typeof entry.folder === "boolean") && validPath(entry.path)); |
| 1661 | + const write = (pubkey, entries) => { |
| 1662 | + const store = storage(); |
| 1663 | + if (!store) return false; |
| 1664 | + try { |
| 1665 | + store.setItem(storageKey(pubkey), JSON.stringify(entries.slice(-MAX_ENTRIES))); |
| 1666 | + return true; |
| 1667 | + } catch { |
| 1668 | + return false; |
| 1669 | + } |
| 1670 | + }; |
| 1671 | + const entryFor = (hash, pubkey) => { |
| 1672 | + const key = actorKey(pubkey); |
| 1673 | + return HASH.test(String(hash || "")) ? read(key).find(entry => entry.hash === hash) || null : null; |
| 1674 | + }; |
| 1675 | + const formatSize = size => { |
| 1676 | + if (!Number.isSafeInteger(size) || size < 0) return ""; |
| 1677 | + if (size < 1024) return `${size} B`; |
| 1678 | + const units = ["KiB", "MiB", "GiB"]; |
| 1679 | + let value = size, unit = "B"; |
| 1680 | + for (const next of units) { |
| 1681 | + value /= 1024; |
| 1682 | + unit = next; |
| 1683 | + if (value < 1024 || next === units.at(-1)) break; |
| 1684 | + } |
| 1685 | + return `${value >= 10 ? value.toFixed(0) : value.toFixed(1)} ${unit}`; |
| 1686 | + }; |
| 1687 | + const decorate = pubkey => { |
| 1688 | + const section = document.querySelector?.("#file-library[data-pubkey]"); |
| 1689 | + if (!section) return 0; |
| 1690 | + let count = 0; |
| 1691 | + for (const row of section.querySelectorAll?.("tr[data-hash]") || []) { |
| 1692 | + const entry = entryFor(row.getAttribute("data-hash"), pubkey); |
| 1693 | + if (!entry) continue; |
| 1694 | + const link = row.querySelector("td:first-child > a") || row.querySelector("a"); |
| 1695 | + if (link) { |
| 1696 | + link.href = entry.link; |
| 1697 | + link.setAttribute("fx-ignore", ""); |
| 1698 | + const name = link.querySelector("span:last-child") || link; |
| 1699 | + name.textContent = entry.name; |
| 1700 | + } |
| 1701 | + const details = row.querySelector("td:nth-child(2)"); |
| 1702 | + if (details && entry.size !== undefined) { |
| 1703 | + const access = /secret link/i.test(details.textContent || ""); |
| 1704 | + details.textContent = access ? "Secret link" : formatSize(entry.size); |
| 1705 | + if (access) { |
| 1706 | + const size = document.createElement("small"); |
| 1707 | + size.textContent = formatSize(entry.size); |
| 1708 | + details.append(size); |
| 1709 | + } |
| 1710 | + if (entry.type) { |
| 1711 | + const type = document.createElement("small"); |
| 1712 | + type.textContent = entry.type; |
| 1713 | + details.append(type); |
| 1714 | + } |
| 1715 | + } |
| 1716 | + count++; |
| 1717 | + } |
| 1718 | + return count; |
| 1719 | + }; |
| 1720 | + const restore = pubkey => { |
| 1721 | + const current = new URL(location.href); |
| 1722 | + if (!/(?:^|\/)file$/.test(current.pathname)) return null; |
| 1723 | + const hash = current.searchParams.get("hash") || current.searchParams.get("sha"); |
| 1724 | + const fragment = new URLSearchParams(current.hash.slice(1)); |
| 1725 | + if (!HASH.test(hash || "") || fragment.get("key") || fragment.get("iv")) return null; |
| 1726 | + const entry = entryFor(hash, pubkey); |
| 1727 | + if (!entry) return null; |
| 1728 | + const saved = new URL(entry.link); |
| 1729 | + if (saved.searchParams.get("hash") !== hash && saved.searchParams.get("sha") !== hash) return null; |
| 1730 | + history.replaceState?.(null, "", saved.pathname + saved.search + saved.hash); |
| 1731 | + return entry; |
| 1732 | + }; |
| 1733 | + const save = (value, pubkey) => { |
| 1734 | + const key = actorKey(pubkey); |
| 1735 | + if (!key || !validEntry(value, key)) return false; |
| 1736 | + const entries = read(key).filter(entry => entry.hash !== value.hash); |
| 1737 | + entries.push({ |
| 1738 | + hash: value.hash, |
| 1739 | + link: value.link, |
| 1740 | + name: value.name, |
| 1741 | + ...(value.type === undefined ? {} : {type: value.type}), |
| 1742 | + ...(value.size === undefined ? {} : {size: value.size}), |
| 1743 | + ...(value.folder === undefined ? {} : {folder: value.folder}), |
| 1744 | + ...(value.path === undefined ? {} : {path: value.path}), |
| 1745 | + savedAt: Date.now() |
| 1746 | + }); |
| 1747 | + return write(key, entries); |
| 1748 | + }; |
| 1749 | + |
| 1750 | + const api = Object.freeze({save, get: entryFor, restore, decorate}); |
| 1751 | + globalThis.tiny = globalThis.tiny || {}; |
| 1752 | + globalThis.tiny.files = {...(globalThis.tiny.files || {}), catalog: api}; |
| 1753 | + const sync = () => { |
| 1754 | + const surface = document.querySelector?.("#file-library[data-pubkey], #file-detail[data-pubkey], #file-view[data-pubkey]"); |
| 1755 | + const actor = surface?.getAttribute("data-pubkey"); |
| 1756 | + if (!actor) return; |
| 1757 | + restore(actor); |
| 1758 | + decorate(actor); |
| 1759 | + }; |
| 1760 | + sync(); |
| 1761 | + document.addEventListener?.("tiny:navigation", sync); |
| 1762 | +})(); |
| 1763 | diff --git a/internal/webui/file-workspace.js b/internal/webui/file-workspace.js |
| 1764 | index a7a57f1..0759f02 100644 |
| 1765 | --- a/internal/webui/file-workspace.js |
| 1766 | +++ b/internal/webui/file-workspace.js |
| 1767 | @@ -15,6 +15,13 @@ |
| 1768 | const hex = value => globalThis.tiny.util.hex(value); |
| 1769 | const fromHex = value => globalThis.tiny.util.fromHex(value || "", "Invalid hash or encryption key."); |
| 1770 | const element = globalThis.tiny.util.element; |
| 1771 | + const fileSize = value => { |
| 1772 | + let size = Number(value) || 0; |
| 1773 | + for (const unit of ["B", "KiB", "MiB", "GiB"]) { |
| 1774 | + if (size < 1024 || unit === "GiB") return (unit === "B" ? size : size.toFixed(1)) + " " + unit; |
| 1775 | + size /= 1024; |
| 1776 | + } |
| 1777 | + }; |
| 1778 | const checkAbort = signal => { |
| 1779 | if (signal?.aborted) |
| 1780 | throw Object.assign(Error("Upload canceled. Retry while this page stays open."), {name: "AbortError"}); |
| 1781 | @@ -55,27 +62,37 @@ |
| 1782 | return {root, name: commonRoot ? paths[0][0] : "folder"}; |
| 1783 | } |
| 1784 | |
| 1785 | - async function storeEncrypted(plaintext, type, signal) { |
| 1786 | + async function storeEncrypted(plaintext, type, signal, session) { |
| 1787 | checkAbort(signal); |
| 1788 | const encrypted = await encryption().encryptCHK(plaintext); |
| 1789 | checkAbort(signal); |
| 1790 | + if (session?.completed.has(encrypted.hash)) return session.completed.get(encrypted.hash); |
| 1791 | const url = new URL(tiny.localPath("/"), location.href).href; |
| 1792 | const result = await upload().upload(encrypted.ciphertext, { |
| 1793 | url, |
| 1794 | hash: encrypted.hash, |
| 1795 | type, |
| 1796 | signal, |
| 1797 | + metadata: {purpose: "chunk"}, |
| 1798 | + retries: 3, |
| 1799 | + state: session?.states.get(encrypted.hash) || (() => { |
| 1800 | + const state = new Map(); |
| 1801 | + session?.states.set(encrypted.hash, state); |
| 1802 | + return state; |
| 1803 | + })(), |
| 1804 | authorize: (target, method, body) => tiny.authorization(target, method, body) |
| 1805 | }); |
| 1806 | if (result.descriptor.sha256 !== encrypted.hash || result.descriptor.size !== encrypted.ciphertext.length) |
| 1807 | throw Error("Relay returned an unexpected blob descriptor."); |
| 1808 | - return {hash: encrypted.hash, key: fromHex(encrypted.key)}; |
| 1809 | + const stored = {hash: encrypted.hash, key: fromHex(encrypted.key)}; |
| 1810 | + session?.completed.set(encrypted.hash, stored); |
| 1811 | + return stored; |
| 1812 | } |
| 1813 | |
| 1814 | const manifestStore = |
| 1815 | - signal => |
| 1816 | + (signal, session) => |
| 1817 | async ({bytes, node}) => |
| 1818 | - storeEncrypted(bytes, node.t === 2 || node.t === 3 ? directoryType : "application/octet-stream", signal); |
| 1819 | + storeEncrypted(bytes, node.t === 2 || node.t === 3 ? directoryType : "application/octet-stream", signal, session); |
| 1820 | const rootReference = details => { |
| 1821 | const stored = details.manifests.at(-1); |
| 1822 | return { |
| 1823 | @@ -86,13 +103,13 @@ |
| 1824 | }; |
| 1825 | }; |
| 1826 | |
| 1827 | - async function storeFile(file, signal, progress) { |
| 1828 | + async function storeFile(file, signal, progress, session) { |
| 1829 | if (file.size > MAX_FILE_BYTES) throw Error("Files must be at most 256 MiB in the browser workspace."); |
| 1830 | const chunks = []; |
| 1831 | for (let offset = 0; offset < file.size || offset === 0; offset += codec().CHUNK_SIZE) { |
| 1832 | checkAbort(signal); |
| 1833 | const plain = new Uint8Array(await file.slice(offset, offset + codec().CHUNK_SIZE).arrayBuffer()); |
| 1834 | - const stored = await storeEncrypted(plain, "application/octet-stream", signal); |
| 1835 | + const stored = await storeEncrypted(plain, "application/octet-stream", signal, session); |
| 1836 | chunks.push({...stored, size: plain.length}); |
| 1837 | progress?.(plain.length); |
| 1838 | if (file.size === 0) break; |
| 1839 | @@ -100,16 +117,16 @@ |
| 1840 | if (chunks.length === 1) return {...chunks[0], type: 0}; |
| 1841 | return rootReference( |
| 1842 | await codec().buildFile(chunks, { |
| 1843 | - store: manifestStore(signal), |
| 1844 | + store: manifestStore(signal, session), |
| 1845 | returnDetails: true |
| 1846 | }) |
| 1847 | ); |
| 1848 | } |
| 1849 | |
| 1850 | - async function storeDirectory(directory, signal, progress) { |
| 1851 | + async function storeDirectory(directory, signal, progress, session) { |
| 1852 | const entries = []; |
| 1853 | for (const [name, file] of directory.files) { |
| 1854 | - const stored = await storeFile(file, signal, progress); |
| 1855 | + const stored = await storeFile(file, signal, progress, session); |
| 1856 | entries.push({ |
| 1857 | ...stored, |
| 1858 | name, |
| 1859 | @@ -117,12 +134,12 @@ |
| 1860 | }); |
| 1861 | } |
| 1862 | for (const [name, child] of directory.directories) { |
| 1863 | - const stored = await storeDirectory(child, signal, progress); |
| 1864 | + const stored = await storeDirectory(child, signal, progress, session); |
| 1865 | entries.push({...stored, name}); |
| 1866 | } |
| 1867 | return rootReference( |
| 1868 | await codec().buildDirectory(entries, { |
| 1869 | - store: manifestStore(signal), |
| 1870 | + store: manifestStore(signal, session), |
| 1871 | returnDetails: true |
| 1872 | }) |
| 1873 | ); |
| 1874 | @@ -157,11 +174,19 @@ |
| 1875 | if (this.bound) return; |
| 1876 | this.bound = true; |
| 1877 | this.innerHTML = |
| 1878 | - '<form><label>Files <input type="file" name="file" multiple></label><input type="file" name="folder" webkitdirectory multiple hidden><p data-drop><button type="button" data-pick-folder>Choose folder</button> or drop files or a folder here</p><label><input type="checkbox" name="encrypt"> encrypt</label><button>Upload</button></form><p data-controls hidden><button type="button" data-cancel disabled>Cancel</button> <button type="button" data-retry disabled>Retry</button></p><label data-share hidden>Share link <input data-share-url readonly></label><output role="status"></output>'; |
| 1879 | + '<form><label>Files <input type="file" name="file" multiple></label><input type="file" name="folder" webkitdirectory multiple hidden><p data-drop><button type="button" data-pick-folder>Choose folder</button> or drop files or a folder here</p><label>Who can open <select name="access"><option value="public">Anyone with the link</option><option value="members">Relay members</option></select></label><details data-advanced><summary>Advanced</summary><label><input type="checkbox" name="encrypt"> Encrypt with a secret link</label><p><small>Only people with the complete link can decrypt it. Saved in this browser for your account.</small></p><label data-background hidden><input type="checkbox" name="background"> Continue in the background</label></details><button>Upload</button></form><progress max="1" value="0" hidden aria-label="Upload progress"></progress><output role="status"></output><p data-controls hidden><button type="button" data-cancel disabled>Cancel</button> <button type="button" data-retry disabled>Retry</button></p><div data-share hidden><p><a data-open>Open file</a> <button type="button" data-copy-link>Copy secret link</button></p><label>Secret link <input data-share-url readonly></label></div>'; |
| 1880 | this.out = this.querySelector("output"); |
| 1881 | const form = this.querySelector("form"); |
| 1882 | const files = form.elements.namedItem("file"); |
| 1883 | const folder = form.elements.namedItem("folder"); |
| 1884 | + const access = form.elements.namedItem("access"); |
| 1885 | + access.value = "public"; |
| 1886 | + access.addEventListener("change", () => { |
| 1887 | + const encrypt = form.elements.namedItem("encrypt"); |
| 1888 | + if (access.value === "members") encrypt.checked = false; |
| 1889 | + encrypt.disabled = access.value === "members"; |
| 1890 | + }); |
| 1891 | + this.querySelector("[data-background]").hidden = !backgroundAvailable(); |
| 1892 | files.addEventListener("change", () => this.choose([...files.files])); |
| 1893 | folder.addEventListener("change", () => this.choose([...folder.files])); |
| 1894 | this.querySelector("[data-pick-folder]").addEventListener("click", () => folder.click()); |
| 1895 | @@ -181,6 +206,7 @@ |
| 1896 | }); |
| 1897 | this.querySelector("[data-cancel]").addEventListener("click", () => this.controller?.abort()); |
| 1898 | this.querySelector("[data-retry]").addEventListener("click", () => this.run().catch(() => {})); |
| 1899 | + this.querySelector("[data-copy-link]").addEventListener("click", () => this.copyLink()); |
| 1900 | this.intake().catch(error => this.say(error.message, true)); |
| 1901 | this.resumeBackground().catch(() => {}); |
| 1902 | } |
| 1903 | @@ -210,7 +236,7 @@ |
| 1904 | ]; |
| 1905 | } |
| 1906 | const id = "tiny-upload-" + crypto.randomUUID(); |
| 1907 | - localStorage.setItem("tiny.bg." + id, JSON.stringify({hash, fragment: fragment || "", name, at: Date.now()})); |
| 1908 | + localStorage.setItem("tiny.bg." + id, JSON.stringify({hash, fragment: fragment || "", name, size: bytes.byteLength - (fragment ? 16 : 0), pubkey: this.getAttribute("pubkey") || "", root: tiny.root || "", at: Date.now()})); |
| 1909 | const task = await registration.backgroundFetch.fetch(id, requests, { |
| 1910 | title: "Uploading " + name, |
| 1911 | icons: [ |
| 1912 | @@ -271,8 +297,9 @@ |
| 1913 | localStorage.removeItem(key); |
| 1914 | continue; |
| 1915 | } |
| 1916 | + if (info.pubkey !== (this.getAttribute("pubkey") || "") || info.root !== (tiny.root || "")) continue; |
| 1917 | const task = await registration.backgroundFetch.get(id).catch(() => null); |
| 1918 | - const finish = descriptor => { |
| 1919 | + const finish = async descriptor => { |
| 1920 | if (!info.fragment) { |
| 1921 | this.say("Background upload of " + info.name + " finished."); |
| 1922 | return tiny.navigate?.(location.href); |
| 1923 | @@ -280,6 +307,8 @@ |
| 1924 | const link = new URL(tiny.localPath("/file"), location.href); |
| 1925 | link.search = "?hash=" + descriptor.sha256; |
| 1926 | link.hash = info.fragment; |
| 1927 | + const params = new URLSearchParams(info.fragment); |
| 1928 | + await this.remember(link.href, [{name: info.name, size: info.size || Math.max(0, descriptor.size - 16), type: params.get("type") || "application/octet-stream"}], false); |
| 1929 | return this.showLink(link.href); |
| 1930 | }; |
| 1931 | try { |
| 1932 | @@ -381,6 +410,13 @@ |
| 1933 | if (error) this.out.dataset.error = ""; |
| 1934 | else delete this.out.dataset.error; |
| 1935 | } |
| 1936 | + progress(sent, total, phase = "Uploading") { |
| 1937 | + const fraction = total ? Math.min(1, sent / total) : 0; |
| 1938 | + const meter = this.querySelector("progress"); |
| 1939 | + meter.hidden = false; |
| 1940 | + meter.value = fraction; |
| 1941 | + this.say(phase + " " + Math.round(fraction * 100) + "% — " + fileSize(sent) + " of " + fileSize(total)); |
| 1942 | + } |
| 1943 | controls(busy, completed) { |
| 1944 | const retryable = Boolean(this.selection) && !completed; |
| 1945 | this.querySelector("[data-controls]").hidden = !busy && !retryable; |
| 1946 | @@ -391,31 +427,41 @@ |
| 1947 | if (this.busy) return; |
| 1948 | if (form) { |
| 1949 | if (!this.chosen) this.choose([...form.elements.namedItem("file").files]); |
| 1950 | - this.selection = {...this.chosen, encrypt: Boolean(form.elements.namedItem("encrypt").checked)}; |
| 1951 | + const access = form.elements.namedItem("access").value === "members" ? "members" : "public"; |
| 1952 | + this.selection = {...this.chosen, access, encrypt: access !== "members" && Boolean(form.elements.namedItem("encrypt").checked), background: Boolean(form.elements.namedItem("background").checked)}; |
| 1953 | this.pending = null; |
| 1954 | + this.completedLink = null; |
| 1955 | + this.manifestSession = {completed: new Map(), states: new Map()}; |
| 1956 | + this.plainStates = new Map(); |
| 1957 | } |
| 1958 | - if (!this.selection?.files.length) throw Error("Choose a file or folder first."); |
| 1959 | + if (!this.selection?.files.length) { this.say("Choose a file or folder first.", true); return; } |
| 1960 | this.busy = true; |
| 1961 | this.controller = new AbortController(); |
| 1962 | this.controls(true, false); |
| 1963 | + this.querySelector("[data-share]").hidden = true; |
| 1964 | let completed = false; |
| 1965 | // Keep the phone awake for the length of the upload. |
| 1966 | const wakeLock = await navigator.wakeLock?.request?.("screen").catch(() => null); |
| 1967 | try { |
| 1968 | const {files, folder, encrypt} = this.selection; |
| 1969 | - let link; |
| 1970 | - if (!encrypt) await this.storePlain(files); |
| 1971 | - else if (folder || files.length > 1) link = await this.storeManifest(files, true); |
| 1972 | - else if (files[0].size > LARGE_FILE_BYTES) link = await this.storeManifest(files, false); |
| 1973 | - else link = await this.storeSealed(files[0]); |
| 1974 | - completed = true; |
| 1975 | + let link = this.completedLink; |
| 1976 | + if (!link) { |
| 1977 | + if (!encrypt) await this.storePlain(files); |
| 1978 | + else if (folder || files.length > 1) link = await this.storeManifest(files, true); |
| 1979 | + else if (files[0].size > LARGE_FILE_BYTES) link = await this.storeManifest(files, false); |
| 1980 | + else link = await this.storeSealed(files[0]); |
| 1981 | + } |
| 1982 | if (link) { |
| 1983 | + this.completedLink = link; |
| 1984 | + await this.remember(link, files, folder); |
| 1985 | await this.showLink(link); |
| 1986 | } else { |
| 1987 | this.say("Stored " + files.length + (files.length === 1 ? " file." : " files.")); |
| 1988 | await tiny.navigate?.(location.href); |
| 1989 | } |
| 1990 | - return link; |
| 1991 | + completed = true; |
| 1992 | + if (link && this.catalogSaved) await tiny.navigate?.(location.href); |
| 1993 | + return link || undefined; |
| 1994 | } catch (error) { |
| 1995 | const canceled = this.controller.signal.aborted; |
| 1996 | this.say( |
| 1997 | @@ -427,24 +473,44 @@ |
| 1998 | wakeLock?.release?.().catch?.(() => {}); |
| 1999 | this.busy = false; |
| 2000 | this.controls(false, completed); |
| 2001 | + if (completed) this.querySelector("progress").hidden = true; |
| 2002 | if (completed) this.selection = this.chosen = null; |
| 2003 | } |
| 2004 | } |
| 2005 | async storePlain(files) { |
| 2006 | const query = new URL(location.href).searchParams; |
| 2007 | const parent = (!query.get("view") || query.get("view") === "library") ? query.get("path") || "" : ""; |
| 2008 | + const total = files.reduce((size, file) => size + file.size, 0); |
| 2009 | + let sent = 0; |
| 2010 | for (const [index, file] of files.entries()) { |
| 2011 | checkAbort(this.controller.signal); |
| 2012 | + if (file.size > MAX_FILE_BYTES) throw Error("Files must be at most 256 MiB in the browser workspace."); |
| 2013 | this.say("Uploading " + file.name + " (" + (index + 1) + " of " + files.length + ")…"); |
| 2014 | const bytes = new Uint8Array(await file.arrayBuffer()); |
| 2015 | const type = file.type || "application/octet-stream"; |
| 2016 | const selectedPath = file.webkitRelativePath || file.name; |
| 2017 | const metadata = {filename: file.name, path: parent ? parent + "/" + selectedPath : selectedPath}; |
| 2018 | - if (files.length === 1 && bytes.byteLength >= BACKGROUND_BYTES && backgroundAvailable()) { |
| 2019 | + if (this.selection?.access === "members") metadata.access = "members"; |
| 2020 | + if (this.selection?.background && files.length === 1 && bytes.byteLength >= BACKGROUND_BYTES && backgroundAvailable()) { |
| 2021 | await this.storeBackground(bytes, {hash: await tiny.sha256hex(bytes), type, name: file.name, metadata}); |
| 2022 | continue; |
| 2023 | } |
| 2024 | - await tiny.signedFetch(this.uploadTarget("/upload", metadata), "PUT", bytes, {contentType: type, signal: this.controller.signal}); |
| 2025 | + this.progress(sent, total, "Uploading " + file.name); |
| 2026 | + if (bytes.byteLength >= BACKGROUND_BYTES && upload()?.upload) { |
| 2027 | + this.plainStates ||= new Map(); |
| 2028 | + if (!this.plainStates.has(index)) this.plainStates.set(index, new Map()); |
| 2029 | + await upload().upload(bytes, { |
| 2030 | + url: new URL(tiny.localPath("/"), location.href).href, |
| 2031 | + type, metadata, retries: 3, state: this.plainStates.get(index), signal: this.controller.signal, |
| 2032 | + authorize: (target, method, body) => tiny.authorization(target, method, body), |
| 2033 | + onProgress: progress => this.progress(sent + progress.sent, total, "Uploading " + file.name), |
| 2034 | + onState: state => { if (state.status === "retrying") this.say("Connection interrupted. Retrying " + file.name + "…"); } |
| 2035 | + }); |
| 2036 | + } else { |
| 2037 | + await tiny.signedFetch(this.uploadTarget("/upload", metadata), "PUT", bytes, {contentType: type, signal: this.controller.signal}); |
| 2038 | + } |
| 2039 | + sent += file.size; |
| 2040 | + this.progress(sent, total); |
| 2041 | } |
| 2042 | } |
| 2043 | uploadTarget(target, metadata) { |
| 2044 | @@ -455,7 +521,7 @@ |
| 2045 | async storeSealed(file) { |
| 2046 | if (!this.pending) { |
| 2047 | if (file.size > MAX_FILE_BYTES) throw Error("Encrypted browser uploads are limited to 256 MiB."); |
| 2048 | - this.say("Encrypting…"); |
| 2049 | + this.say("Encrypting " + file.name + " (" + fileSize(file.size) + ")…"); |
| 2050 | const plaintext = new Uint8Array(await file.arrayBuffer()); |
| 2051 | const key = await crypto.subtle.generateKey({name: "AES-GCM", length: 256}, true, ["encrypt", "decrypt"]); |
| 2052 | const iv = crypto.getRandomValues(new Uint8Array(12)); |
| 2053 | @@ -477,11 +543,13 @@ |
| 2054 | checkAbort(this.controller.signal); |
| 2055 | const {ciphertext, fragment, state} = this.pending; |
| 2056 | const hash = await tiny.sha256hex(ciphertext); |
| 2057 | + this.progress(0, file.size); |
| 2058 | let descriptor; |
| 2059 | - if (ciphertext.byteLength >= BACKGROUND_BYTES && backgroundAvailable() && upload()?.plan) { |
| 2060 | + if (this.selection?.background && ciphertext.byteLength >= BACKGROUND_BYTES && backgroundAvailable() && upload()?.plan) { |
| 2061 | descriptor = await this.storeBackground(ciphertext, { |
| 2062 | hash, |
| 2063 | type: "application/octet-stream", |
| 2064 | + metadata: {purpose: "chunk"}, |
| 2065 | name: file.name, |
| 2066 | fragment |
| 2067 | }); |
| 2068 | @@ -490,14 +558,17 @@ |
| 2069 | url: new URL(tiny.localPath("/"), location.href).href, |
| 2070 | hash, |
| 2071 | type: "application/octet-stream", |
| 2072 | + metadata: {purpose: "chunk"}, |
| 2073 | + retries: 3, |
| 2074 | state, |
| 2075 | signal: this.controller.signal, |
| 2076 | authorize: (target, method, body) => tiny.authorization(target, method, body), |
| 2077 | - onProgress: progress => this.say("Uploading " + Math.round(progress.fraction * 100) + "%…") |
| 2078 | + onProgress: progress => this.progress(Math.min(file.size, progress.sent), file.size), |
| 2079 | + onState: state => { if (state.status === "retrying") this.say("Connection interrupted. Retrying " + file.name + "…"); } |
| 2080 | }); |
| 2081 | descriptor = (await task).descriptor; |
| 2082 | } else { |
| 2083 | - const response = await tiny.signedFetch("/upload", "PUT", ciphertext, { |
| 2084 | + const response = await tiny.signedFetch("/upload?purpose=chunk", "PUT", ciphertext, { |
| 2085 | contentType: "application/octet-stream", |
| 2086 | signal: this.controller.signal |
| 2087 | }); |
| 2088 | @@ -512,36 +583,64 @@ |
| 2089 | } |
| 2090 | async storeManifest(files, folder) { |
| 2091 | let sent = 0; |
| 2092 | + const total = files.reduce((size, file) => size + file.size, 0); |
| 2093 | const progress = size => { |
| 2094 | sent += size; |
| 2095 | - this.say("Stored " + sent.toLocaleString() + " bytes. Encrypting the remaining content…"); |
| 2096 | + this.progress(sent, total, "Encrypting and uploading"); |
| 2097 | }; |
| 2098 | - this.say("Encrypting selected content…"); |
| 2099 | + this.progress(0, total, "Encrypting and uploading"); |
| 2100 | + this.manifestSession ||= {completed: new Map(), states: new Map()}; |
| 2101 | let reference, name, type; |
| 2102 | if (folder) { |
| 2103 | const selected = selectionTree(files); |
| 2104 | name = selected.name || "files"; |
| 2105 | - reference = await storeDirectory(selected.root, this.controller.signal, progress); |
| 2106 | + reference = await storeDirectory(selected.root, this.controller.signal, progress, this.manifestSession); |
| 2107 | } else { |
| 2108 | const file = files[0]; |
| 2109 | name = file.name; |
| 2110 | type = file.type; |
| 2111 | - reference = await storeFile(file, this.controller.signal, progress); |
| 2112 | + reference = await storeFile(file, this.controller.signal, progress, this.manifestSession); |
| 2113 | } |
| 2114 | checkAbort(this.controller.signal); |
| 2115 | return shareURL(reference, name, type); |
| 2116 | } |
| 2117 | + async remember(link, files, folder) { |
| 2118 | + const url = new URL(link); |
| 2119 | + const params = new URLSearchParams(url.hash.slice(1)); |
| 2120 | + const hash = url.searchParams.get("hash"); |
| 2121 | + const name = params.get("name") || files[0].name; |
| 2122 | + const type = params.get("type") || files[0].type || "application/octet-stream"; |
| 2123 | + const size = files.reduce((sum, file) => sum + file.size, 0); |
| 2124 | + const pubkey = this.getAttribute("pubkey"); |
| 2125 | + const isFolder = folder || ["2", "3"].includes(params.get("node")); |
| 2126 | + this.catalogSaved = tiny.files?.catalog?.save({hash, link, name, type, size, folder: isFolder}, pubkey) || false; |
| 2127 | + // Keep the working key visible even if recording the library entry fails. |
| 2128 | + await this.showLink(link); |
| 2129 | + if (tiny.signedFetch) { |
| 2130 | + const label = (isFolder ? "Encrypted folder " : "Encrypted file ") + hash.slice(0, 12); |
| 2131 | + const response = await tiny.signedFetch("/files/metadata", "POST", JSON.stringify({hash, name: label, path: label, purpose: "file", size, type: "application/octet-stream"}), {contentType: "application/json", signal: this.controller?.signal}); |
| 2132 | + if (response.ok === false) throw Error("File uploaded; its library entry could not be saved. Retry to finish."); |
| 2133 | + } |
| 2134 | + tiny.files?.catalog?.decorate(pubkey); |
| 2135 | + } |
| 2136 | async showLink(link) { |
| 2137 | const share = this.querySelector("[data-share-url]"); |
| 2138 | share.value = link; |
| 2139 | this.querySelector("[data-share]").hidden = false; |
| 2140 | + const open = this.querySelector("[data-open]"); |
| 2141 | + open.href = link; |
| 2142 | + open.setAttribute("fx-ignore", ""); |
| 2143 | + this.say(this.catalogSaved ? "Stored. You can reopen this file from My files in this browser." : "Stored. Keep the secret link to reopen this encrypted file."); |
| 2144 | + } |
| 2145 | + async copyLink() { |
| 2146 | + const share = this.querySelector("[data-share-url]"); |
| 2147 | try { |
| 2148 | if (!navigator.clipboard?.writeText) throw Error("Clipboard unavailable"); |
| 2149 | - await navigator.clipboard.writeText(link); |
| 2150 | - this.say("Stored. Share link copied; the key is only in its fragment."); |
| 2151 | + await navigator.clipboard.writeText(share.value); |
| 2152 | + this.say("Secret link copied."); |
| 2153 | } catch { |
| 2154 | share.select?.(); |
| 2155 | - this.say("Stored. Copy the share link above; the key is only in its fragment."); |
| 2156 | + this.say("Copy the secret link from the field above."); |
| 2157 | } |
| 2158 | } |
| 2159 | } |
| 2160 | @@ -638,18 +737,19 @@ |
| 2161 | const root = codec().decodeManifest(data); |
| 2162 | if (root.t !== type) throw Error("Manifest type does not match its reference."); |
| 2163 | if (type === 1) { |
| 2164 | - const button = element("button", "Download decrypted file"); |
| 2165 | + const media = /^(video\/(mp4|webm|ogg)|audio\/(mpeg|mp4|ogg|wav|webm))$/.test(params.get("type") || ""); |
| 2166 | + const button = element("button", media ? "Load media player" : "Prepare download"); |
| 2167 | button.type = "button"; |
| 2168 | - button.addEventListener("click", () => |
| 2169 | - this.download({ |
| 2170 | + const open = () => this.download({ |
| 2171 | ...reference, |
| 2172 | t: 1, |
| 2173 | s: root.l.reduce((sum, link) => sum + link.s, 0), |
| 2174 | n: params.get("name") || "file", |
| 2175 | m: {type: params.get("type")} |
| 2176 | - }).catch(error => this.fail(error.message)) |
| 2177 | - ); |
| 2178 | + }).catch(error => this.fail(error.message)); |
| 2179 | + button.addEventListener("click", open); |
| 2180 | this.append(button); |
| 2181 | + if (media) await open(); |
| 2182 | return; |
| 2183 | } |
| 2184 | const entries = await codec().resolveDirectory(root, hash => fetchBlob(hash, this.controller.signal), { |
| 2185 | @@ -676,7 +776,7 @@ |
| 2186 | open.href = url.href; |
| 2187 | item.append(open); |
| 2188 | } else { |
| 2189 | - const button = element("button", link.n + " (" + link.s.toLocaleString() + " bytes)"); |
| 2190 | + const button = element("button", link.n + " (" + fileSize(link.s) + ")"); |
| 2191 | button.type = "button"; |
| 2192 | button.addEventListener("click", () => this.download(link).catch(error => this.fail(error.message))); |
| 2193 | item.append(button); |
| 2194 | @@ -687,11 +787,24 @@ |
| 2195 | this.entries = entries; |
| 2196 | } |
| 2197 | async download(link) { |
| 2198 | + if (this.downloading) return; |
| 2199 | + this.downloading = true; |
| 2200 | + const status = this.querySelector("output") || element("output"); |
| 2201 | + status.setAttribute("role", "status"); |
| 2202 | + if (!status.parentNode) this.append(status); |
| 2203 | + status.textContent = "Loading " + link.n + " (" + fileSize(link.s) + ")…"; |
| 2204 | + try { |
| 2205 | let data; |
| 2206 | if (link.t === 0) data = await plaintext(link, this.controller.signal); |
| 2207 | else if (link.t === 1) { |
| 2208 | const manifest = codec().decodeManifest(await plaintext(link, this.controller.signal)); |
| 2209 | - data = await codec().readFile(manifest, hash => fetchBlob(hash, this.controller.signal), { |
| 2210 | + let fetched = 0; |
| 2211 | + data = await codec().readFile(manifest, async (hash, part) => { |
| 2212 | + const raw = await fetchBlob(hash, this.controller.signal); |
| 2213 | + if (part.t === 0) fetched += part.s; |
| 2214 | + status.textContent = "Loading " + Math.min(100, Math.round(fetched / link.s * 100)) + "% — " + fileSize(fetched) + " of " + fileSize(link.s); |
| 2215 | + return raw; |
| 2216 | + }, { |
| 2217 | decrypt, |
| 2218 | maxDepth: MAX_DEPTH, |
| 2219 | maxBytes: MAX_FILE_BYTES |
| 2220 | @@ -699,7 +812,9 @@ |
| 2221 | } else throw Error("Unsupported file link."); |
| 2222 | if (data.length !== link.s) throw Error("File size does not match its directory entry."); |
| 2223 | this.offerDownload(data, link.n, link.m?.type); |
| 2224 | + status.textContent = "Ready. " + fileSize(data.length) + " decrypted in your browser."; |
| 2225 | return data; |
| 2226 | + } finally { this.downloading = false; } |
| 2227 | } |
| 2228 | offerDownload(data, name, type) { |
| 2229 | if (this.downloadURL) URL.revokeObjectURL(this.downloadURL); |
| 2230 | @@ -707,8 +822,26 @@ |
| 2231 | const link = element("a", "Save " + name); |
| 2232 | link.href = this.downloadURL; |
| 2233 | link.download = name; |
| 2234 | - this.append(link); |
| 2235 | - link.click(); |
| 2236 | + this.querySelector("[data-prepared-file]")?.remove(); |
| 2237 | + const prepared = element("div"); |
| 2238 | + prepared.setAttribute("data-prepared-file", ""); |
| 2239 | + let media; |
| 2240 | + if (/^video\/(mp4|webm|ogg)$/.test(type || "")) media = element("video"); |
| 2241 | + else if (/^audio\/(mpeg|mp4|ogg|wav|webm)$/.test(type || "")) media = element("audio"); |
| 2242 | + else if (/^image\/(png|jpeg|gif|webp|avif)$/.test(type || "")) media = element("img"); |
| 2243 | + if (media) { |
| 2244 | + media.src = this.downloadURL; |
| 2245 | + media.controls = true; |
| 2246 | + media.preload = "metadata"; |
| 2247 | + if (media.tagName === "IMG") media.alt = name; |
| 2248 | + prepared.append(media); |
| 2249 | + } else if (/^(text\/|application\/(json|javascript|xml|x-(javascript|sh|python|ruby))|image\/svg\+xml$)/.test(type || "")) { |
| 2250 | + const preview = element("pre", new TextDecoder().decode(new Uint8Array(data).subarray(0, 256 * 1024))); |
| 2251 | + prepared.append(preview); |
| 2252 | + if (data.byteLength > 256 * 1024) prepared.append(element("p", "Preview shows the first 256 KiB. Download the file to read it all.")); |
| 2253 | + } |
| 2254 | + prepared.append(link); |
| 2255 | + this.append(prepared); |
| 2256 | } |
| 2257 | } |
| 2258 | customElements.define("file-upload", FileUpload); |
| 2259 | diff --git a/internal/webui/page.html b/internal/webui/page.html |
| 2260 | index 0ff1aea..261d277 100644 |
| 2261 | --- a/internal/webui/page.html |
| 2262 | +++ b/internal/webui/page.html |
| 2263 | @@ -75,6 +75,7 @@ then feature modules are loaded for the current page. */}} |
| 2264 | <script src="{{scriptURL "tiny.js"}}"></script> |
| 2265 | <script src="{{scriptURL "bridge.js"}}"></script> |
| 2266 | {{if or (eq .Tab "files") (eq .Tab "file")}}<script src="{{scriptURL "blossom-encryption.js"}}"></script> |
| 2267 | +<script src="{{scriptURL "file-catalog.js"}}"></script> |
| 2268 | <script src="{{scriptURL "blossom-manifests.js"}}"></script> |
| 2269 | <script src="{{scriptURL "blossom-upload.js"}}"></script> |
| 2270 | <script src="{{scriptURL "file-messages.js"}}"></script> |
| 2271 | @@ -129,7 +130,7 @@ then feature modules are loaded for the current page. */}} |
| 2272 | <section><h4>Publish</h4><p><small>Announce with ngit and push with plain git. Every push needs the owner's signed state.</small></p></section>{{end}} |
| 2273 | |
| 2274 | {{define "panel-files"}}<section><h4>Storage</h4><table><tr><th>protocol</th><td>Blossom</td></tr>{{if .Policy.Features.Files}}<tr><th>uploads</th><td>members</td></tr>{{end}}</table></section> |
| 2275 | -{{if .Actor}}<section><h4>Upload</h4><file-upload></file-upload><p><small>Encrypt keeps the contents and key in your browser. Keep the complete share link.</small></p><details><summary>Import from URL</summary><file-mirror action="/mirror"></file-mirror></details><noscript><p>Uploads need JavaScript and a connected signer.</p></noscript></section>{{end}}{{end}} |
| 2276 | +{{if .Actor}}<section><h4>Upload</h4><file-upload pubkey="{{.Actor}}"></file-upload><p><small>Members-only files require membership to view or download.</small></p><details><summary>Import from URL</summary><file-mirror action="/mirror"></file-mirror></details><noscript><p>Uploads need JavaScript and a connected signer.</p></noscript></section>{{end}}{{end}} |
| 2277 | |
| 2278 | {{define "panel-approvals"}}{{$counts := approvalCounts .Event}}{{$data := asMap .Event}}<section><h4>Waiting</h4><table><tr><th>open</th><td>{{index $counts "open"}}</td></tr><tr><th>oldest</th><td>{{with index $data "oldest_open"}}<time datetime="{{datetime .}}">{{when .}}</time>{{else}}none{{end}}</td></tr><tr><th>answered</th><td>{{index $counts "answered"}}</td></tr><tr><th>expired</th><td>{{index $counts "expired"}}</td></tr></table></section> |
| 2279 | <section><h4>Devices</h4>{{if .Actor}}{{range $index, $device := approvalDevices .Event}}<p><i></i>device {{add $index 1}} | {{$device}}</p>{{else}}<p>No device receives notifications for this key. Enable them under Inbox.</p>{{end}}<p><small>Answers from the phone need the remembered signer.</small></p>{{else}}<p>Sign in to see which devices are notified.</p>{{end}}</section>{{end}} |
| 2280 | diff --git a/internal/webui/style.css b/internal/webui/style.css |
| 2281 | index 99efb90..4fab4e8 100644 |
| 2282 | --- a/internal/webui/style.css |
| 2283 | +++ b/internal/webui/style.css |
| 2284 | @@ -138,6 +138,13 @@ file-tools h3,file-upload h3{margin:0 0 4px}file-tools p,file-upload p{margin:6p |
| 2285 | rpc-form label,signed-form label,publish-list label,nostr-compose label,file-tools label,file-upload label,#collaboration-filter label,#file-search label,#wiki-search label{display:inline-block;margin:0 10px 6px 0;vertical-align:top;font:13px var(--sans);color:var(--muted)} |
| 2286 | rpc-form label input,rpc-form label select,signed-form label input,nostr-compose label input,nostr-compose label select,file-tools label input,file-tools label select,file-upload label input,#collaboration-filter label input,#collaboration-filter label select,#file-search label input,#wiki-search label input{display:block;margin-top:2px;font:13px var(--mono);min-width:200px} |
| 2287 | file-upload label[data-share]{display:block}file-upload input[data-share-url]{width:100%} |
| 2288 | +file-upload form>label,file-upload [data-share] label{display:block;max-width:100%;margin-right:0} |
| 2289 | +file-upload select,file-upload progress{display:block;width:100%;max-width:100%;margin:6px 0 10px} |
| 2290 | +file-upload progress{height:8px;accent-color:var(--accent)} |
| 2291 | +file-upload input[type=file],file-upload input[data-share-url]{min-width:0;max-width:100%;box-sizing:border-box} |
| 2292 | +file-workspace-root video,file-workspace-root img,file-tools video,file-tools img{display:block;width:auto;max-width:100%;max-height:70vh;margin:12px 0} |
| 2293 | +file-workspace-root video,file-tools video{width:100%;background:#000} |
| 2294 | +file-workspace-root audio,file-tools audio{display:block;max-width:100%;margin:12px 0} |
| 2295 | file-upload label:has([type=checkbox]),#panel file-upload label:has([type=checkbox]){display:inline-block;margin:8px 14px 8px 0}file-upload label:has([type=checkbox]) input,#panel file-upload label:has([type=checkbox]) input{display:inline-block;width:auto;min-width:0;margin:0 4px 0 0;vertical-align:middle} |
| 2296 | #panel file-upload [hidden],file-upload [hidden],file-tools [hidden],#panel push-toggle [hidden]{display:none} |
| 2297 | #panel push-toggle label{display:inline-block;margin:6px 12px 0 0}#panel push-toggle label input{display:inline-block;width:auto;min-width:0;margin:0 4px 0 0;vertical-align:middle}#panel push-toggle form{margin-top:4px} |
| 2298 | diff --git a/internal/webui/templates.go b/internal/webui/templates.go |
| 2299 | index 2c7dbe0..bf44417 100644 |
| 2300 | --- a/internal/webui/templates.go |
| 2301 | +++ b/internal/webui/templates.go |
| 2302 | @@ -65,6 +65,9 @@ var blossomUploadJS string |
| 2303 | //go:embed file-workspace.js |
| 2304 | var fileWorkspaceJS string |
| 2305 | |
| 2306 | +//go:embed file-catalog.js |
| 2307 | +var fileCatalogJS string |
| 2308 | + |
| 2309 | // Installable app assets. The manifest is a template: NAME and BASE are |
| 2310 | // replaced per request so tenant prefixes and relay names stay correct. |
| 2311 | // |
| 2312 | @@ -222,7 +225,7 @@ func repoView(query url.Values) string { |
| 2313 | // scripts are served at /scripts/<name> and cached by the service worker. |
| 2314 | // One version stamp covers them all, so a change to any file refreshes every |
| 2315 | // cached copy together. |
| 2316 | -var scripts = map[string]string{"bridge.js": bridgeJS, "tiny.js": tinyJS, "components.js": componentsJS, "rooms.js": roomsJS, "nostr-name.js": nostrNameJS, "blossom-encryption.js": blossomEncryptionJS, "blossom-manifests.js": blossomManifestsJS, "blossom-upload.js": blossomUploadJS, "file-messages.js": fileMessagesJS, "private-services.js": privateServicesJS, "file-workspace.js": fileWorkspaceJS} |
| 2317 | +var scripts = map[string]string{"bridge.js": bridgeJS, "tiny.js": tinyJS, "components.js": componentsJS, "rooms.js": roomsJS, "nostr-name.js": nostrNameJS, "blossom-encryption.js": blossomEncryptionJS, "blossom-manifests.js": blossomManifestsJS, "blossom-upload.js": blossomUploadJS, "file-messages.js": fileMessagesJS, "private-services.js": privateServicesJS, "file-workspace.js": fileWorkspaceJS, "file-catalog.js": fileCatalogJS} |
| 2318 | |
| 2319 | var scriptsVersion = func() string { |
| 2320 | names := make([]string, 0, len(scripts)) |
| 2321 | diff --git a/internal/webui/tiny.js b/internal/webui/tiny.js |
| 2322 | index 5a7def6..bad624f 100644 |
| 2323 | --- a/internal/webui/tiny.js |
| 2324 | +++ b/internal/webui/tiny.js |
| 2325 | @@ -141,7 +141,7 @@ |
| 2326 | // Feature bundles load on the pages that use them and again after an |
| 2327 | // in-place navigation lands on such a page. Each script loads once. |
| 2328 | tiny.bundles = { |
| 2329 | - files: ["blossom-encryption.js", "blossom-manifests.js", "blossom-upload.js", "file-messages.js", "file-workspace.js"], |
| 2330 | + files: ["file-catalog.js", "blossom-encryption.js", "blossom-manifests.js", "blossom-upload.js", "file-messages.js", "file-workspace.js"], |
| 2331 | private: ["private-services.js"] |
| 2332 | }; |
| 2333 | const loaded = new Map(); |
| 2334 | diff --git a/scripts/blossom-upload.test.mjs b/scripts/blossom-upload.test.mjs |
| 2335 | index e678e93..df8aff0 100644 |
| 2336 | --- a/scripts/blossom-upload.test.mjs |
| 2337 | +++ b/scripts/blossom-upload.test.mjs |
| 2338 | @@ -27,6 +27,7 @@ test("uses BUD-14 PATCH chunks and signs each chunk payload", async () => { |
| 2339 | hash, |
| 2340 | type: "text/plain", |
| 2341 | chunkSize: 4, |
| 2342 | + probeCache: false, |
| 2343 | fetch: async (url, init) => { |
| 2344 | calls.push({url, init}); |
| 2345 | return calls.length === 1 ? response(204, {allow: "GET, PUT, PATCH"}) : |
| 2346 | @@ -52,6 +53,7 @@ test("falls back to BUD-13 PUT when PATCH is not advertised", async () => { |
| 2347 | url: "https://blossom.example/upload", |
| 2348 | hash, |
| 2349 | type: "text/plain", |
| 2350 | + probeCache: false, |
| 2351 | fetch: async (url, init) => { |
| 2352 | calls.push({url, init}); |
| 2353 | return calls.length === 1 ? response(204, {allow: "GET, PUT"}) : response(201, {}, {sha256: hash, size: bytes.length}); |
| 2354 | @@ -69,7 +71,7 @@ test("retries transient chunks, reports local state, and can be canceled", async |
| 2355 | let attempts = 0; |
| 2356 | const states = []; |
| 2357 | const task = upload(bytes, { |
| 2358 | - url: "https://blossom.example/upload", hash, chunkSize: 2, retries: 1, |
| 2359 | + url: "https://blossom.example/upload", hash, chunkSize: 2, retries: 1, probeCache: false, |
| 2360 | fetch: async (url, init) => { |
| 2361 | if (init.method === "OPTIONS") return response(204, {allow: "PATCH"}); |
| 2362 | attempts++; |
| 2363 | @@ -110,6 +112,25 @@ test("resumes recent local chunks, rejects stale state, and checks the descripto |
| 2364 | }), /invalid Blossom upload descriptor/); |
| 2365 | }); |
| 2366 | |
| 2367 | +test("caches endpoint capability and appends upload metadata", async () => { |
| 2368 | + const bytes = new Uint8Array([1, 2]); |
| 2369 | + const hash = await digest(bytes); |
| 2370 | + const calls = []; |
| 2371 | + const fetcher = async (url, init) => { |
| 2372 | + calls.push({url, init}); |
| 2373 | + return init.method === "OPTIONS" ? response(204, {allow: "PATCH"}) : response(201, {}, {sha256: hash, size: bytes.length}); |
| 2374 | + }; |
| 2375 | + await upload(bytes, {url: "https://metadata.example/upload", hash, chunkSize: bytes.length, fetch: fetcher, |
| 2376 | + metadata: {access: "members", purpose: "file", filename: "secret.bin"}}); |
| 2377 | + await upload(bytes, {url: "https://metadata.example/upload", hash, chunkSize: bytes.length, fetch: fetcher, |
| 2378 | + metadata: {access: "members", purpose: "chunk", filename: "secret.bin"}}); |
| 2379 | + assert.equal(calls.filter(call => call.init.method === "OPTIONS").length, 1); |
| 2380 | + const patch = calls.find(call => call.init.method === "PATCH"); |
| 2381 | + assert.match(patch.url, /access=members/); |
| 2382 | + assert.match(patch.url, /purpose=file/); |
| 2383 | + assert.match(patch.url, /filename=secret.bin/); |
| 2384 | +}); |
| 2385 | + |
| 2386 | test("keeps the response body readable after headers arrive", async () => { |
| 2387 | const bytes = new Uint8Array([11, 12]); |
| 2388 | const hash = await digest(bytes); |
| 2389 | diff --git a/scripts/file-catalog.test.mjs b/scripts/file-catalog.test.mjs |
| 2390 | new file mode 100644 |
| 2391 | index 0000000..33b285e |
| 2392 | --- /dev/null |
| 2393 | +++ b/scripts/file-catalog.test.mjs |
| 2394 | @@ -0,0 +1,50 @@ |
| 2395 | +import test from "node:test"; |
| 2396 | +import assert from "node:assert/strict"; |
| 2397 | +import fs from "node:fs"; |
| 2398 | +import vm from "node:vm"; |
| 2399 | + |
| 2400 | +const source = fs.readFileSync("internal/webui/file-catalog.js", "utf8"); |
| 2401 | +const pubkey = "a".repeat(64); |
| 2402 | +const hash = "b".repeat(64); |
| 2403 | + |
| 2404 | +function setup({path = "/r/test/files", actor = pubkey} = {}) { |
| 2405 | + const values = new Map(); |
| 2406 | + const rows = []; |
| 2407 | + const document = { |
| 2408 | + querySelector(selector) { |
| 2409 | + if (selector === "[data-pubkey]") return {getAttribute: () => actor}; |
| 2410 | + if (selector === "#file-library") return {querySelectorAll: () => rows}; |
| 2411 | + return null; |
| 2412 | + }, |
| 2413 | + createElement() { return {textContent: "", append() {}}; } |
| 2414 | + }; |
| 2415 | + const location = {href: "https://relay.test" + path, origin: "https://relay.test", pathname: path}; |
| 2416 | + const history = {replaceState(_state, _title, value) { location.href = "https://relay.test" + value; location.pathname = new URL(location.href).pathname; location.hash = new URL(location.href).hash; }}; |
| 2417 | + const context = { |
| 2418 | + location, history, document, |
| 2419 | + localStorage: {getItem: key => values.get(key) || null, setItem: (key, value) => values.set(key, value)}, |
| 2420 | + URL, URLSearchParams, Date, Number, String, Array, Boolean, Object, RegExp, JSON, |
| 2421 | + globalThis: null, tiny: {localPath: value => "/r/test" + value} |
| 2422 | + }; |
| 2423 | + context.globalThis = context; |
| 2424 | + vm.runInNewContext(source, context); |
| 2425 | + return {context, values, rows}; |
| 2426 | +} |
| 2427 | + |
| 2428 | +const link = () => `https://relay.test/r/test/file?hash=${hash}#key=${"c".repeat(64)}&iv=${"d".repeat(16)}&name=video.mp4&type=video%2Fmp4`; |
| 2429 | + |
| 2430 | +test("saves encrypted metadata per tenant and account and restores it after reload", () => { |
| 2431 | + const first = setup({path: "/r/test/file?hash=" + hash}); |
| 2432 | + assert.equal(first.context.tiny.files.catalog.save({hash, link: link(), name: "video.mp4", type: "video/mp4", size: 104857600}, pubkey), true); |
| 2433 | + assert.equal(first.context.tiny.files.catalog.get(hash, pubkey).name, "video.mp4"); |
| 2434 | + assert.equal(first.context.tiny.files.catalog.get(hash, "e".repeat(64)), null); |
| 2435 | + assert.equal(first.context.tiny.files.catalog.restore(pubkey).name, "video.mp4"); |
| 2436 | + assert.match(first.context.location.href, /#key=/); |
| 2437 | +}); |
| 2438 | + |
| 2439 | +test("rejects unsafe links, names and cross-tenant links", () => { |
| 2440 | + const {context} = setup(); |
| 2441 | + assert.equal(context.tiny.files.catalog.save({hash, link: "https://evil.test/r/test/file?hash=" + hash + "#key=x", name: "x"}, pubkey), false); |
| 2442 | + assert.equal(context.tiny.files.catalog.save({hash, link: link(), name: "../secret"}, pubkey), false); |
| 2443 | + assert.equal(context.tiny.files.catalog.save({hash, link: link().replace("/r/test/", "/r/other/"), name: "x"}, pubkey), false); |
| 2444 | +}); |
| 2445 | diff --git a/scripts/file-media-preview.test.mjs b/scripts/file-media-preview.test.mjs |
| 2446 | new file mode 100644 |
| 2447 | index 0000000..e256598 |
| 2448 | --- /dev/null |
| 2449 | +++ b/scripts/file-media-preview.test.mjs |
| 2450 | @@ -0,0 +1,141 @@ |
| 2451 | +import test from "node:test"; |
| 2452 | +import assert from "node:assert/strict"; |
| 2453 | +import fs from "node:fs"; |
| 2454 | +import vm from "node:vm"; |
| 2455 | + |
| 2456 | +class Node { |
| 2457 | + constructor(tag = "") { this.tagName = tag.toUpperCase(); this.children = []; this.attributes = {}; this.listeners = {}; this.textContent = ""; this.dataset = {}; } |
| 2458 | + append(...nodes) { for (const node of nodes) if (node && typeof node !== "string") { this.children.push(node); node.parentNode = this; } } |
| 2459 | + remove() { this.parentNode?.children.splice(this.parentNode.children.indexOf(this), 1); } |
| 2460 | + setAttribute(name, value) { this.attributes[name] = String(value); } |
| 2461 | + getAttribute(name) { return this.attributes[name] ?? null; } |
| 2462 | + hasAttribute(name) { return Object.hasOwn(this.attributes, name); } |
| 2463 | + addEventListener(name, fn) { (this.listeners[name] ||= []).push(fn); } |
| 2464 | + click() {} |
| 2465 | + replaceChildren(...nodes) { this.children = []; this.append(...nodes); } |
| 2466 | + querySelector(selector) { return this.querySelectorAll(selector)[0] || null; } |
| 2467 | + querySelectorAll(selector) { |
| 2468 | + const selectors = selector.split(",").map(value => value.trim()); |
| 2469 | + const match = node => selectors.some(value => { |
| 2470 | + const attr = value.match(/^([^[]+)?\[([^=\]]+)(?:="([^"]+)")?\]$/); |
| 2471 | + if (attr) return (!attr[1] || node.tagName === attr[1].toUpperCase()) && node.hasAttribute(attr[2]) && (attr[3] === undefined || node.getAttribute(attr[2]) === attr[3]); |
| 2472 | + return node.tagName === value.toUpperCase(); |
| 2473 | + }); |
| 2474 | + const found = []; |
| 2475 | + const walk = node => { for (const child of node.children) { if (match(child)) found.push(child); walk(child); } }; |
| 2476 | + walk(this); return found; |
| 2477 | + } |
| 2478 | + set innerHTML(value) { |
| 2479 | + this.children = []; |
| 2480 | + const stack = [this]; |
| 2481 | + for (const token of value.match(/<[^>]+>|[^<]+/g) || []) { |
| 2482 | + if (!token.startsWith("<")) continue; |
| 2483 | + if (/^<\//.test(token)) { stack.pop(); continue; } |
| 2484 | + const match = token.match(/^<([\w-]+)([^>]*)>/); if (!match) continue; |
| 2485 | + const child = new Node(match[1]); |
| 2486 | + for (const [, name, quote, attrValue] of match[2].matchAll(/([\w-]+)(?:=(['"])(.*?)\2)?/g)) child.setAttribute(name, attrValue ?? ""); |
| 2487 | + stack.at(-1).append(child); |
| 2488 | + if (!/^(input|br|meta|img)$/.test(match[1])) stack.push(child); |
| 2489 | + } |
| 2490 | + } |
| 2491 | +} |
| 2492 | + |
| 2493 | +const setup = () => { |
| 2494 | + const document = new Node("document"); |
| 2495 | + document.readyState = "complete"; |
| 2496 | + document.documentElement = new Node("html"); |
| 2497 | + document.body = new Node("body"); |
| 2498 | + document.createElement = name => new Node(name); |
| 2499 | + document.addEventListener = () => {}; |
| 2500 | + document.dispatchEvent = () => {}; |
| 2501 | + document.querySelectorAll = () => []; |
| 2502 | + const customElements = {registry: {}, define(name, ctor) { this.registry[name] = ctor; }}; |
| 2503 | + const location = {href: "https://relay.test/file?hash=cipher", origin: "https://relay.test", pathname: "/file", hash: ""}; |
| 2504 | + const window = {document, customElements, location, addEventListener: () => {}, tiny: {util: {}, blossom: {encryption: {}}}}; |
| 2505 | + class HTMLElement extends Node {} |
| 2506 | + Object.assign(globalThis, {window, document, customElements, HTMLElement, location, Blob, URL, CustomEvent: class {}}); |
| 2507 | + globalThis.tiny = window.tiny; |
| 2508 | + vm.runInThisContext(fs.readFileSync("internal/webui/tiny.js", "utf8")); |
| 2509 | + window.tiny.localPath = path => "/r/test" + path; |
| 2510 | + window.tiny.require = async () => {}; |
| 2511 | + window.tiny.sha256hex = async () => "cipher"; |
| 2512 | + window.tiny.blossom.encryption.decryptCHK = async () => new TextEncoder().encode("video bytes"); |
| 2513 | + vm.runInThisContext(fs.readFileSync("internal/webui/components.js", "utf8")); |
| 2514 | + return {FileTools: customElements.registry["file-tools"], window}; |
| 2515 | +}; |
| 2516 | + |
| 2517 | +const makeTool = (FileTools, window, type = "video/mp4") => { |
| 2518 | + window.location.hash = "#enc=chk-v1&key=secret&name=clip.mp4&type=" + encodeURIComponent(type); |
| 2519 | + const tool = new FileTools(); |
| 2520 | + tool.setAttribute("hash", "cipher"); |
| 2521 | + tool.setAttribute("type", "application/octet-stream"); |
| 2522 | + return tool; |
| 2523 | +}; |
| 2524 | + |
| 2525 | +test("encrypted MP4 renders a local video preview and replaces it on retry", async () => { |
| 2526 | + const {FileTools, window} = setup(); |
| 2527 | + const revoked = []; |
| 2528 | + const originalCreate = URL.createObjectURL, originalRevoke = URL.revokeObjectURL; |
| 2529 | + URL.createObjectURL = blob => "blob:preview-" + blob.type + "-" + Math.random(); |
| 2530 | + URL.revokeObjectURL = url => revoked.push(url); |
| 2531 | + const originalFetch = globalThis.fetch; |
| 2532 | + globalThis.fetch = async () => new Response(new Uint8Array([1, 2, 3])); |
| 2533 | + try { |
| 2534 | + const tool = makeTool(FileTools, window); tool.connectedCallback(); |
| 2535 | + await new Promise(resolve => setTimeout(resolve, 0)); |
| 2536 | + const first = tool.querySelector("[data-decrypted-preview]"); |
| 2537 | + assert.ok(first); |
| 2538 | + assert.equal(first.querySelector("video").controls, true); |
| 2539 | + assert.equal(first.querySelector("video").preload, "metadata"); |
| 2540 | + assert.equal(first.querySelector("video").src.startsWith("blob:preview-video/mp4-"), true); |
| 2541 | + await tool.decryptFromFragment(); |
| 2542 | + assert.equal(tool.querySelectorAll("[data-decrypted-preview]").length, 1); |
| 2543 | + assert.equal(revoked.length, 1, "retry releases the previous object URL"); |
| 2544 | + tool.disconnectedCallback(); |
| 2545 | + assert.equal(revoked.length, 2, "disconnect releases the current object URL"); |
| 2546 | + } finally { |
| 2547 | + globalThis.fetch = originalFetch; |
| 2548 | + URL.createObjectURL = originalCreate; |
| 2549 | + URL.revokeObjectURL = originalRevoke; |
| 2550 | + } |
| 2551 | +}); |
| 2552 | + |
| 2553 | +test("HTML and SVG render as inert bounded source text", async () => { |
| 2554 | + for (const type of ["image/svg+xml", "text/html"]) { |
| 2555 | + const {FileTools, window} = setup(); |
| 2556 | + const originalFetch = globalThis.fetch; |
| 2557 | + globalThis.fetch = async () => new Response(new Uint8Array([1, 2, 3])); |
| 2558 | + try { |
| 2559 | + const tool = makeTool(FileTools, window, type); tool.connectedCallback(); |
| 2560 | + await tool.decryptFromFragment(); |
| 2561 | + const preview = tool.querySelector("[data-decrypted-preview]"); |
| 2562 | + assert.ok(preview, type + " should have a source preview"); |
| 2563 | + assert.equal(preview.querySelector("pre").textContent, "video bytes"); |
| 2564 | + assert.equal(preview.querySelector("img"), null); |
| 2565 | + assert.equal(preview.querySelector("iframe"), null); |
| 2566 | + assert.ok(tool.querySelector("a[download]") || tool.querySelector("a")); |
| 2567 | + } finally { globalThis.fetch = originalFetch; } |
| 2568 | + } |
| 2569 | +}); |
| 2570 | + |
| 2571 | +test("plain text preview is bounded while download remains complete", async () => { |
| 2572 | + const {FileTools, window} = setup(); |
| 2573 | + const originalFetch = globalThis.fetch; |
| 2574 | + const originalDecrypt = window.tiny.blossom.encryption.decryptCHK; |
| 2575 | + const source = "A".repeat(256 * 1024 + 20); |
| 2576 | + window.tiny.blossom.encryption.decryptCHK = async () => new TextEncoder().encode(source); |
| 2577 | + globalThis.fetch = async () => new Response(new Uint8Array([1, 2, 3])); |
| 2578 | + try { |
| 2579 | + const tool = makeTool(FileTools, window, "text/plain"); tool.connectedCallback(); |
| 2580 | + await new Promise(resolve => setTimeout(resolve, 0)); |
| 2581 | + const pre = tool.querySelector("pre"); |
| 2582 | + assert.ok(pre); |
| 2583 | + assert.equal(pre.textContent.startsWith("A".repeat(256 * 1024)), true); |
| 2584 | + assert.match(pre.textContent, /Preview truncated/); |
| 2585 | + const downloaded = await (await originalFetch(tool.downloadURL)).text(); |
| 2586 | + assert.equal(downloaded, source); |
| 2587 | + } finally { |
| 2588 | + window.tiny.blossom.encryption.decryptCHK = originalDecrypt; |
| 2589 | + globalThis.fetch = originalFetch; |
| 2590 | + } |
| 2591 | +}); |
| 2592 | diff --git a/scripts/file-workspace.test.mjs b/scripts/file-workspace.test.mjs |
| 2593 | index 61c102c..1e8effe 100644 |
| 2594 | --- a/scripts/file-workspace.test.mjs |
| 2595 | +++ b/scripts/file-workspace.test.mjs |
| 2596 | @@ -9,6 +9,7 @@ class Node { |
| 2597 | append(...nodes) { nodes.forEach(node => { if (node) { this.children.push(node); node.parentNode = this; } }); } |
| 2598 | replaceChildren(...nodes) { this.children = []; this.append(...nodes); } |
| 2599 | click() {} |
| 2600 | + remove() { if (this.parentNode) this.parentNode.children = this.parentNode.children.filter(child => child !== this); } |
| 2601 | setAttribute(name, value) { this.attributes[name] = String(value); } |
| 2602 | getAttribute(name) { return this.attributes[name] ?? null; } |
| 2603 | hasAttribute(name) { return name in this.attributes; } |
| 2604 | @@ -120,7 +121,7 @@ const sealedUpload = async (window, FileUpload, file) => { |
| 2605 | delete window.tiny.blossom.upload; |
| 2606 | window.tiny.localPath = path => path; |
| 2607 | window.tiny.sha256hex = async bytes => Buffer.from(await webcrypto.subtle.digest("SHA-256", bytes)).toString("hex"); |
| 2608 | - window.tiny.signedFetch = async (_path, _method, body) => { uploaded = new Uint8Array(body); return {json: async () => ({sha256: await window.tiny.sha256hex(body)})}; }; |
| 2609 | + window.tiny.signedFetch = async (_path, _method, body) => { if (_path === "/files/metadata") return {ok: true}; uploaded = new Uint8Array(body); return {json: async () => ({sha256: await window.tiny.sha256hex(body)})}; }; |
| 2610 | const upload = new FileUpload(); window.document.append(upload); upload.connectedCallback(); |
| 2611 | const form = upload.querySelector("form"); |
| 2612 | form.elements.namedItem("file").files = [file]; |
| 2613 | @@ -191,14 +192,20 @@ test("folder upload rejects unexpected server hashes", async t => { |
| 2614 | await assert.rejects(server.store([folderFile("private", "root/private.txt")]), /descriptor/); |
| 2615 | }); |
| 2616 | |
| 2617 | -test("encrypted upload keeps ciphertext and link when clipboard fails", async () => { |
| 2618 | +test("encrypted upload never touches the clipboard until Copy is pressed", async () => { |
| 2619 | const {window, FileUpload} = load(true); |
| 2620 | + let copies = 0; |
| 2621 | + navigator.clipboard.writeText = async () => { copies++; throw Error("denied"); }; |
| 2622 | const {upload, link, uploaded} = await sealedUpload(window, FileUpload, new File(["secret plaintext"], "secret.txt", {type: "text/plain"})); |
| 2623 | assert.notEqual(new TextDecoder().decode(uploaded()), "secret plaintext"); |
| 2624 | assert.match(link, /\/file\?hash=[0-9a-f]{64}#key=.*&iv=/); |
| 2625 | assert.equal(upload.querySelector("[data-share-url]").value, link); |
| 2626 | assert.equal(upload.querySelector("[data-share]").hidden, false); |
| 2627 | - assert.match(upload.out.textContent, /Copy the share link above/); |
| 2628 | + assert.equal(copies, 0); |
| 2629 | + assert.equal(upload.querySelector("[data-open]").href, link); |
| 2630 | + await upload.copyLink(); |
| 2631 | + assert.equal(copies, 1); |
| 2632 | + assert.match(upload.out.textContent, /Copy the secret link from the field/); |
| 2633 | }); |
| 2634 | |
| 2635 | test("plain uploads store every chosen file and refresh the listing", async () => { |
| 2636 | @@ -308,6 +315,110 @@ test("retry keeps the original ciphertext and key after interrupted encrypted up |
| 2637 | assert.equal(upload.pending, null); |
| 2638 | }); |
| 2639 | |
| 2640 | +test("manifest retry keeps completed chunks and reports readable progress", async t => { |
| 2641 | + const server = workspaceServer(t); |
| 2642 | + const transport = globalThis.fetch; |
| 2643 | + let interrupted = true; |
| 2644 | + const attempts = new Map(); |
| 2645 | + globalThis.fetch = async (url, init = {}) => { |
| 2646 | + if (init.method === "PATCH") { |
| 2647 | + const hash = new URL(url).pathname.split("/").at(-1); |
| 2648 | + attempts.set(hash, (attempts.get(hash) || 0) + 1); |
| 2649 | + if (interrupted && attempts.size === 2) return new Response(null, {status: 403}); |
| 2650 | + } |
| 2651 | + return transport(url, init); |
| 2652 | + }; |
| 2653 | + const bytes = new Uint8Array(5 * 1024 * 1024); |
| 2654 | + bytes[0] = 1; bytes[2 * 1024 * 1024] = 2; bytes[4 * 1024 * 1024] = 3; |
| 2655 | + const upload = new server.FileUpload(); upload.connectedCallback(); |
| 2656 | + const form = upload.querySelector("form"); |
| 2657 | + form.elements.namedItem("file").files = [folderFile(bytes, "video/clip.mp4")]; |
| 2658 | + form.elements.namedItem("encrypt").checked = true; |
| 2659 | + await assert.rejects(upload.run(form), /403/); |
| 2660 | + const first = attempts.keys().next().value; |
| 2661 | + assert.equal(upload.manifestSession.completed.size, 1); |
| 2662 | + interrupted = false; |
| 2663 | + const link = await upload.run(); |
| 2664 | + assert.equal(attempts.get(first), 1, "accepted first chunk must not be reuploaded"); |
| 2665 | + const root = await server.open(link); |
| 2666 | + assert.deepEqual(await root.download(root.entries[0]), bytes); |
| 2667 | + upload.progress(3 * 1024 * 1024, 5 * 1024 * 1024); |
| 2668 | + assert.match(upload.out.textContent, /60%.*3.0 MiB.*5.0 MiB/); |
| 2669 | + assert.equal(upload.querySelector("progress").value, 0.6); |
| 2670 | + for (const request of server.requests.filter(r => r.init.method === "PATCH")) { |
| 2671 | + assert.equal(request.url.searchParams.get("purpose"), "chunk"); |
| 2672 | + } |
| 2673 | +}); |
| 2674 | + |
| 2675 | +test("members upload keeps the MP4 name and type and never creates a secret link", async () => { |
| 2676 | + const {window, FileUpload} = load(true); |
| 2677 | + const sent = []; |
| 2678 | + window.tiny.signedFetch = async (path, method, bytes, options) => { sent.push({path, method, bytes, options}); }; |
| 2679 | + window.tiny.navigate = async () => {}; |
| 2680 | + const upload = new FileUpload(); upload.connectedCallback(); |
| 2681 | + const form = upload.querySelector("form"); |
| 2682 | + form.elements.namedItem("file").files = [new File(["mp4 bytes"], "holiday.mp4", {type: "video/mp4"})]; |
| 2683 | + form.elements.namedItem("access").value = "members"; |
| 2684 | + form.elements.namedItem("encrypt").checked = true; |
| 2685 | + await upload.run(form); |
| 2686 | + assert.equal(sent.length, 1); |
| 2687 | + const url = new URL(sent[0].path, location.href); |
| 2688 | + assert.equal(url.searchParams.get("access"), "members"); |
| 2689 | + assert.equal(url.searchParams.get("filename"), "holiday.mp4"); |
| 2690 | + assert.equal(sent[0].options.contentType, "video/mp4"); |
| 2691 | + assert.equal(new TextDecoder().decode(sent[0].bytes), "mp4 bytes"); |
| 2692 | + assert.equal(upload.querySelector("[data-share]").hidden, true); |
| 2693 | +}); |
| 2694 | + |
| 2695 | +test("retry finalizes the same encrypted root without reuploading or exposing its key", async () => { |
| 2696 | + const {window, FileUpload} = load(true); |
| 2697 | + window.tiny.localPath = path => path; |
| 2698 | + window.tiny.sha256hex = async bytes => Buffer.from(await webcrypto.subtle.digest("SHA-256", bytes)).toString("hex"); |
| 2699 | + let uploads = 0, records = 0; |
| 2700 | + window.tiny.blossom.upload = {upload: async (bytes, options) => { |
| 2701 | + uploads++; |
| 2702 | + assert.equal(options.metadata.purpose, "chunk"); |
| 2703 | + return {descriptor: {sha256: await window.tiny.sha256hex(bytes), size: bytes.length}}; |
| 2704 | + }}; |
| 2705 | + window.tiny.signedFetch = async (path, method, body) => { |
| 2706 | + assert.equal(path, "/files/metadata"); |
| 2707 | + assert.equal(method, "POST"); |
| 2708 | + const record = JSON.parse(body); |
| 2709 | + assert.match(record.name, /^Encrypted file [a-f0-9]{12}$/); |
| 2710 | + assert.equal(record.purpose, "file"); |
| 2711 | + assert.equal(record.size, 7); |
| 2712 | + assert.equal(body.includes("secret-name.txt"), false); |
| 2713 | + assert.equal("key" in record, false); |
| 2714 | + if (++records === 1) throw Error("server restarting"); |
| 2715 | + return {ok: true}; |
| 2716 | + }; |
| 2717 | + const upload = new FileUpload(); upload.connectedCallback(); |
| 2718 | + const form = upload.querySelector("form"); |
| 2719 | + form.elements.namedItem("file").files = [new File(["private"], "secret-name.txt")]; |
| 2720 | + form.elements.namedItem("encrypt").checked = true; |
| 2721 | + await assert.rejects(upload.run(form), /server restarting/); |
| 2722 | + const saved = upload.querySelector("[data-share-url]").value; |
| 2723 | + assert.match(saved, /#key=/); |
| 2724 | + assert.equal(upload.querySelector("[data-retry]").disabled, false); |
| 2725 | + assert.equal(await upload.run(), saved); |
| 2726 | + assert.equal(uploads, 1); |
| 2727 | + assert.equal(records, 2); |
| 2728 | +}); |
| 2729 | + |
| 2730 | +test("a 100 MiB encrypted MP4 opens a media player after reconstruction", async t => { |
| 2731 | + const server = workspaceServer(t); |
| 2732 | + const bytes = new Uint8Array(100 * 1024 * 1024); |
| 2733 | + for (let index = 0; index < 50; index++) bytes[index * 2 * 1024 * 1024] = index; |
| 2734 | + const link = await server.store([new File([bytes], "large.mp4", {type: "video/mp4"})]); |
| 2735 | + const root = await server.open(link); |
| 2736 | + const video = root.querySelector("video"); |
| 2737 | + assert.ok(video, "decrypted manifest MP4 must have a player"); |
| 2738 | + assert.equal(video.controls, true); |
| 2739 | + const data = await server.originalFetch(video.src).then(response => response.arrayBuffer()); |
| 2740 | + assert.deepEqual(new Uint8Array(data), bytes); |
| 2741 | + assert.match(root.querySelector("output").textContent, /100.0 MiB/); |
| 2742 | +}); |
| 2743 | + |
| 2744 | test("large uploads hand signed requests to Background Fetch and read the parked descriptor", async () => { |
| 2745 | const {window, FileUpload} = load(true); |
| 2746 | window.tiny.localPath = path => path; |
| 2747 | @@ -334,6 +445,7 @@ test("large uploads hand signed requests to Background Fetch and read the parked |
| 2748 | const upload = new FileUpload(); window.document.append(upload); upload.connectedCallback(); |
| 2749 | const form = upload.querySelector("form"); |
| 2750 | form.elements.namedItem("file").files = [new File([bytes], "big.bin", {type: "application/octet-stream"})]; |
| 2751 | + form.elements.namedItem("background").checked = true; |
| 2752 | const run = upload.run(form); |
| 2753 | await new Promise(resolve => setTimeout(resolve, 50)); |
| 2754 | assert.ok(started, "background fetch was not started"); |