fix: out of memory errors for large number of tables #4523
Conversation
- Closes PostgREST#4462 - Closes PostgREST#4463 Signed-off-by: Taimoor Zaeem <[email protected]>
|
I tested this manually using According to my analysis, max |
| tableNotFoundHint :: Text -> Text -> [Table] -> Maybe Text | ||
| tableNotFoundHint schema tblName tblList | ||
| = fmap (\tbl -> "Perhaps you meant the table '" <> schema <> "." <> tbl <> "'") perhapsTable | ||
| | length tblList <= 4000 = fmap (\tbl -> "Perhaps you meant the table '" <> schema <> "." <> tbl <> "'") perhapsTable |
There was a problem hiding this comment.
There are several problems with this change:
length tblList complexity is linear in the list size and it has to iterate the whole list even if it is longer than 4k.
Leaving it like this opens up a DoS vulnerability.
4k is too many from my tests. On my machine PostgREST became unstable with 1k tables.
This is another DoS vulnerability.
Re-creating FuzzySet upon each request opens up yet another DoS vulnerability.
I'm afraid there is really no other sensible solution to this than #4472
There was a problem hiding this comment.
4k is too many from my tests. On my machine PostgREST became unstable with 1k tables.
Yes, agreed. I was proposing 500 to be the upper limit as mentioned in #4523 (comment).
2 participants
ServerDisconnectedErrorwhen querying tables created after PostgREST startup since v13+ #4463As discussed in #4468 (comment).