267 Commits

Author SHA1 Message Date
Apoorv Mishra
79829a3129 Ability to create share url slug (#4550)
* feat: share url slug

* feat: add col urlId

* feat: allow updating urlId

* fix: typo

* fix: migrations

* fix: urlId model validation

* fix: input label

* fix: debounce slug request

* feat: link preview

* fix: send slug variant in response if available

* fix: temporary redirect to slug variant if available

* fix: move up the custom link field

* fix: process and display backend err

* fix: reset custom link state on popover close and remove isCopied

* fix: document link preview

* fix: set urlId when available

* fix: keep unique(urlId, teamId)

* fix: codeql

* fix: get rid of preview type

* fix: width not needed for block elem

* fix: migrations

* fix: array not required

* fix: use val

* fix: validation on shareId and test

* fix: allow clearing urlId

* fix: do not escape

* fix: unique error text

* fix: keep team
2022-12-13 17:26:36 -08:00
Tom Moor
0f19c550f9 fix: Uploaded and immediately deleted images are not removed from storage (#4562)
* fix: Uploaded and immediately deleted images are not removed from storage upon permanant delete
closes #4557

* Move attachment deletion async
2022-12-11 08:29:38 -08:00
Tom Moor
549b7ab030 tsc 2022-12-06 09:32:13 -05:00
Tom Moor
98d79e1e8b perf: Improve collab persistence performance (#4544)
* stash

* Remove query of state in documentCollaborativeUpdater
2022-12-06 05:21:10 -08:00
Tom Moor
e15ad530de tsc 2022-12-03 09:25:53 -05:00
Denis
6354acca85 if OIDC provider gets user_id as integer Postgres failed query (#4527) 2022-12-02 21:46:56 -08:00
Tom Moor
83b687a632 fix: teamPermanentDeleter execution order 2022-11-30 23:07:37 -05:00
Apoorv Mishra
a6125be6f1 Introduce zod for server-side validations (#4397)
* chore(server): use zod for validations

* fix(server): use ctx.input for documents.list

* fix(server): schema for documents.archived

* fix(server): documents.deleted, documents.viewed & documents.drafts

* fix(server): documents.info

* fix(server): documents.export & documents.restore

* fix(server): documents.search_titles & documents.search

* fix(server): documents.templatize

* fix(server): replace nullish() with optional()

* fix(server): documents.update

* fix(server): documents.move

* fix(server): remaining

* fix(server): add validation for snippet min and max words

* fix(server): fix update types

* fix(server): remove DocumentSchema

* fix(server): collate duplicate schemas

* fix: typos

* fix: reviews

* chore: Fixed case of Metrics import

* fix: restructure /api

* fix: loosen validation for id as it can be a slug too

* Add test for query by slug
Simplify import

Co-authored-by: Tom Moor <tom.moor@gmail.com>
2022-11-24 10:11:43 +05:30
Tom Moor
622f464b9f Store import<->document relationship (#4415)
* Store import<->document relationship

* Update 20221112152649-import-document-relationship.js

* Store importId on collection, UI tweaks on import screen
2022-11-12 08:22:41 -08:00
Tom Moor
cff67f4ca7 fix: Self-hosted logic for allowed domains (#4412)
* fix: Self-hosted logic for allowed domains

* test
2022-11-11 19:19:46 -08:00
Tom Moor
6788005115 Add missing team->user constraint (#4411)
* Add missing team->user constraint

* fix: teamPermanentDeleter cannot complete when team has domains
2022-11-11 09:40:52 -08:00
Tom Moor
587f062677 Remove usage of tiley (#4406)
* First pass

* Mooarrr

* lint

* snapshots
2022-11-08 17:12:22 -08:00
Tom Moor
f2fb5dd1e5 fix: Team creation with private avatar. Do not attempt to copy, closes #4378 2022-11-03 08:58:33 -04:00
Tom Moor
f6f90ff406 fix: Unable to login with matching email from another auth provider (#4356)
* fix: Unable to login with matching email from another auth provider

* refactor
2022-10-29 13:46:29 -07:00
Apoorv Mishra
a89d30c735 Allow drafts to be created without requiring a collection (#4175)
* feat(server): allow document to be created without collectionId

* fix(server): policies for a draft doc without collection

* fix(app): hide share button for drafts

* feat(server): permissions around publishing a draft

* fix(server): return drafts without collection

* fix(server): handle draft deletion

* fix(server): show drafts in deleted docs

* fix(server): allow drafts without collection to be restored

* feat(server): return drafts in search results

* fix: use buildDraftDocument for drafts

* fix: remove isDraftWithoutCollection

* fix: do not return drafts for team

* fix: put invariants

* fix: query clause

* fix: check only for undefined

* fix: restore includeDrafts clause as it was before
2022-10-25 18:01:57 +05:30
Nan Yu
39fc8d5c14 feat: allow ad-hoc creation of new teams (#3964)
Co-authored-by: Tom Moor <tom@getoutline.com>
2022-10-16 05:57:27 -07:00
Tom Moor
1fbc000e03 chore: Reduce test boilerplate (#4300)
* chore: Reduce test boilerplate

* mo
2022-10-15 19:40:21 -07:00
Tom Moor
1915a453db fix: Disallow adding self to collection (#4299)
* api

* ui

* update collection permissions
2022-10-15 19:11:09 -07:00
Tom Moor
7bac696eaf fix #4294 2022-10-15 14:06:56 -04:00
Tom Moor
7199088d1b fix: Multiplayer changes attributed to incorrect user (#4282)
* fix: Multiplayer changes attributed to the wrong user, performance improvements

* fix: Actually use _last_ editor
2022-10-12 06:19:07 -07:00
Tom Moor
933fbb2578 feat: Option for separate edit mode (#4203)
* stash

* wip

* cleanup

* Remove collaborativeEditing toggle, it will always be on in next release.
Flip separateEdit -> seamlessEdit

* Clarify language, hide toggle when collaborative editing is disabled

* Flip boolean to match, easier to reason about
2022-10-02 08:58:33 -07:00
Tom Moor
91d8d27f2d feat: Render diffs in email notifications (#4164)
* deps

* diffCompact

* Diffs in email

* test

* fix: Fade deleted images
fix: Don't include empty paragraphs as context
fix: Allow for same image multiple times and refactor

* Remove target _blank

* fix: Table heading incorrect color
2022-09-24 14:29:11 -07:00
Tom Moor
e8a6de3f18 feat: Add HTML export option (#4056)
* tidy

* Add title to HTML export

* fix: Add compatability for documents without collab state

* Add HTML download option to UI

* docs

* fix nodes that required document to render

* Refactor to allow for styling of HTML export

* div>article for easier programatic content extraction
2022-09-07 04:34:39 -07:00
Tom Moor
32b76303e5 Add simple count of views to share links (#4036)
* Add simple count of views to share links

* Remove no longer applicable tests

* Avoid incrementing view count for known bots
2022-08-30 23:16:40 -07:00
CuriousCorrelation
24c71c38a5 feat: Document subscriptions (#3834)
Co-authored-by: Tom Moor <tom.moor@gmail.com>
2022-08-25 23:47:13 -07:00
Tom Moor
f32f07cdcc chore: Refactor user activation to command 2022-08-18 11:24:27 +02:00
Apoorv Mishra
0c51bfb899 perf: reduce memory usage upon running server tests (#3949)
* perf: reduce memory usage upon running server tests

* perf: plug leaks in server/routes

* perf: plug leaks in server/scripts

* perf: plug leaks in server/policies

* perf: plug leaks in server/models

* perf: plug leaks in server/middlewares

* perf: plug leaks in server/commands

* fix: missing await on db.flush

* perf: plug leaks in server/queues

* chore: remove unused legacy funcs

* fix: await on db.flush

* perf: await on GC to run in between tests

* fix: remove db refs

* fix: revert embeds

* perf: plug leaks in shared/i18n
2022-08-11 21:39:17 +05:30
Tom Moor
1d242d44b1 chore: Add eslint rule for object shorthand (#3955) 2022-08-11 05:18:14 -07:00
Tom Moor
ea28dc46eb fix: Error in WebhookProcessor when team is permanatly destroyed 2022-07-26 22:33:48 +01:00
Tom Moor
71da57773e docs 2022-07-24 14:09:43 +01:00
Tom Moor
ec35af4bc5 Refactor validations 2022-07-24 13:40:04 +01:00
Nan Yu
870d9ed41e feat: allow external SSO methods to log into teams as long as emails match (#3813)
* wip

* wip

* fix comments

* better separation of conerns

* fix up tests

* fix semantics

* fixup tsc

* fix some tests

* the old semantics were easier to use

* add db:reset to scripts

* explicitly throw for unauthorized external authorization

* fix minor bug

* add additional tests for user creator and team creator

* yank the email matching logic out of teamcreator

* renaming

* fix type and test errors

* adds test to ensure that accountProvisioner works with email matching

* remove only

* fix comments

* recreate changes to allow self hosted to make teams
2022-07-24 04:55:30 -07:00
Tom Moor
7ae892fe06 fix: Long collection description prevents import (#3847)
* fix: Long collection description prevents import
fix: Parallelize attachment upload during import

* fix: Improve Notion image import matching

* chore: Bump JSZIP (perf)

* fix: Allow redirect from /doc/<id> to canonical url

* fix: Importing document with only title duplicates title in body
2022-07-24 01:37:20 -07:00
Tom Moor
ef0412c449 fix: Cannot create new team on self-hosted (#3819) 2022-07-20 13:18:21 -07:00
Nan Yu
c3f5563e7f feat: scope login attempts to specific subdomains if available - do not switch subdomains (#3741)
* make the user lookup in user creator sensitive to team
* add team specific logic to oidc strat
* factor out slugifyDomain
* change type of req during auth to Koa.Context
2022-07-19 06:50:55 -07:00
Tom Moor
e509719c77 Add ability to quickly create test users in development (#3764) 2022-07-15 16:11:30 -07:00
Tom Moor
47e73cee4e feat: Cleanup api keys and webhooks for suspended users (#3756) 2022-07-13 00:59:31 -07:00
Nan Yu
1e808fc52c Feat: add auth provider to users on sign in (#3739)
* feat: merge a new authentication method onto existing user records when emails match

* adds test for invite acceptance and auth provider creation

* addresses comments
- test existing user and invites in different test cases
- update lastActiveAt syncronously when an invite is accepted

* sort arrays in test to prevent nondeterministic test behaivior when doing array compare
2022-07-08 00:24:46 -07:00
Tom Moor
c36e7bfbb6 fix: Add 10 domain limit per team (#3733)
* fix: Validate team domains are FQDN's
Add 10 domain limit per team
fix: Deletion of domains not happening within request lifecycle

* tests

* docs
2022-07-05 12:27:02 -07:00
Tom Moor
62d9bf7105 chore: Move initial avatar upload to background worker (#3727)
* chore: Async user avatar upload processor

* chore: Async team avatar upload

* Refactor to task for retries

* Docs
Include avatarUrl in task props to prevent race condition
Remove transaction around upload fetch request
2022-07-03 02:36:15 -07:00
Tom Moor
0c30d2bb34 fix: share.document can be null when document is deleted
closes #3724
2022-07-02 19:56:15 +02:00
Tom Moor
863f22750f feat: Add optional notification email when invite is accepted (#3718)
* feat: Add optional notification email when invite is accepted

* Refactor to use beforeSend
2022-07-02 05:40:40 -07:00
Tom Moor
c9cd424a8d chore: Remove over-usage of invariant (#3719) 2022-07-02 05:29:39 -07:00
Tom Moor
10f86ed218 feat: Webhooks (#3691)
* Webhooks (#3607)

* Get the migration and the model setup. Also make the sample env file a bit easier to use. Now just requires setting a SECRET_KEY and besides that will boot up from the sample

* WIP: Start getting a Webhook page created. Just the skeleton state right now

* WIP: Getting a form created to create webhooks, need to bring in react-hook-forms now

* WIP: Get library installed and make TS happy

* Get a few checkboxes ready to go

* Get creating and destroying working with a decent start to a frontend

* Didn't mean to enable this

* Remove eslint and fix other random typescript issue

* Rename some events to be more realistic

* Revert these changes

* PR review comments around policies. Also make sure this inherits from IdModel so it actually gets an id

* Allow any admin on the team to edit webhooks

* Start sending some webhooks for some User events

* Make sure the URL is valid

* Start recording webhook deliveries

* Make sure to verify if the subscription is for the type of event we are looking at

* Refactor sending Webhooks and follow better webhook schema

This creates a presenter to unify the format of webhooks. We also
extract the sending of webhooks and recording their deliveries to a
method than can be used by each of the different event type methods

We also add a status to WebhookDelivery since we need to save the record
before we make the HTTP request to get its id. Then once we make the
request and get a response we can update the delivery with the HTTP info

* Turn off a subscription that has failed for the last 25 deliveries

* Get a first spec passing. Found a bug in my returning of promises so good to patch that up now

* This looks nicer

* Get some tests added for the processor

* Add cron task to delete older webhooks

* Add Document Events to the Processor

* Revisions, FileOperations and Collections

* Get all the server side events added to the processor and make Typescript make sure they are all accounted for

* Get all the events added to the Frontend and work on styling them a bit, still needs some love though

* Get UI styled up a bit

* Get events wired up for webhook subscriptions

* Get delete events working and test at least one variant of them

* Get deletes working and actually make sure to send the model id in the webhook

* Remove webhook secrets from this slice

* Add disabled label for subscriptions that are disabled

* Make sure to cascade the delete

* Reorg this file a bit

* Fix association

* I removed secret for the moment

* Apply Copy changes from PR Review

Co-authored-by: Tom Moor <tom.moor@gmail.com>

* Actually apply the copy changes

TIL that if you Resolve a conversation it _also_ removes the 'staged suggestion' from your list on Github

Co-authored-by: Tom Moor <tom.moor@gmail.com>

* Update app/scenes/Settings/Webhooks.tsx

Missed this copy change before

Co-authored-by: Tom Moor <tom.moor@gmail.com>

* Add disabled as yellow badge

* Resolve frontend comments

* Fixup Schema a bit and remove the dependency on the subscription

* Add test to make sure we don't disable until there are enough failures, and fix code to actually do that. Also some test fixes from the json response shape changes

* Fix WebhookDeliveries to store the responses as Text instead of blobs

* Switch to text better for response bodies, this is using the helpers better and makes the code read better

* Move the logic to a task but run in through the processor cause the tests expect that right now, moving the tests over next

* Split up the tests and actually enqueue the events from the WebhookProcessor instead of doing them inline

* Allow any team admin to see any webhook subscription for the team

* Add the indexes based on our lookup patterns

* Run eslint --fix to fix auto correct issues from when I tried to use Github to merge copy changes

* Allow subscriptions to be edited after creation

* Types caught that I didn't add the new event to the webhook processor, also added it to the frontend here

* I think this will get these into the translations file

* Catch a few more translations, use styled components better and remove usage of webhook subscription in the copy

Co-authored-by: Tom Moor <tom.moor@gmail.com>

* fix: tsc
fix: Document model payload empty

* fix: Revision webhook payload
Add custom UA for hooks

* Add webhooks icon, move under Integrations settings
Some spacing fixes

* Add actorId to webhook payloads

* Add View and ApiKey event types

* Spacing tweaks, fix team payload

* fix: Webhook not disabled after 25 failures

* fix: Enable webhook when editing if previously disabled

* fix: Correctly store response headers

* fix: Error in json/parsing/presentation results in hanging 'pending' webhook delivery

* fix: Awkward payload for users.invite webhook

* Add BaseEvent, ShareEvent

* fix: Add share events to form

* fix: Move webhook delivery cleanup to single DB call
Remove some unused abstraction

* Add user, collection, group context to membership webhook events
Some associated refactoring

Co-authored-by: Corey Alexander <coreyja@gmail.com>
2022-06-28 22:44:50 -07:00
Tom Moor
68ce304b48 fix: Language in document notification email, missing collection name 2022-06-24 10:01:54 +02:00
Tom Moor
6d4da176d1 chore: Move provisionSubdomain from Team model to teamCreator command 2022-06-22 11:09:20 +02:00
Tom Moor
305de71e8b chore: Block all email providers from being added as team domains (#3678) 2022-06-21 01:29:43 -07:00
Tom Moor
4b27feff61 fix: Enable documents.update with collab editing (#3647)
* fix: Enable documents.update with collab editing

* jest cannot deal with ESM deps
2022-06-20 06:36:25 -07:00
Tom Moor
aed8d7a649 fix: SSR meta data for nested shared documents (#3646) 2022-06-08 01:38:34 -07:00
Tom Moor
728790e38f feat: Validate Google, Azure, OIDC SSO access (#3590)
* chore: Store expiresAt on UserAuthentications. This represents the time that the accessToken is no longer valid and should be exchanged using the refreshToken

* feat: Check and expire Google SSO

* fix: Better handling of multiple auth methods
Added more docs

* fix: Retry access validation with network errors

* Small refactor, add Azure token validation support

* doc

* test

* lint

* OIDC refresh support

* CheckSSOAccessTask -> ValidateSSOAccessTask
Added lastValidatedAt column
Skip checks if validated within 5min
Some edge cases around encrypted columns
2022-06-05 13:18:51 -07:00