Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
50 commits
Select commit Hold shift + click to select a range
418e209
Add --skip-bootstrap to avoid contacting of seed nodes on startup (us…
Doy-lee Apr 3, 2025
7ceff8a
Respect the skip_bootstrap_ flag
Doy-lee Apr 28, 2025
8d18d4b
Get rid of db getter and setter by making it public
Doy-lee May 16, 2025
dae553a
WIP: Return more extensive SN data ready payload
Doy-lee May 16, 2025
c545a1c
Move swarm logs into derivation function, serialise SNDataReady w/ he…
Doy-lee May 20, 2025
89ceced
Timestamps in the DB are stored in millisecond precision
Doy-lee Jun 2, 2025
6522329
Represent pending/ready members w/ an enum
Doy-lee Jun 2, 2025
0b4532e
Remove unused std::optional header
Doy-lee Jun 2, 2025
8460381
Store network swarm/swarm state to disk to resume from
Doy-lee Jun 4, 2025
7ee669c
Update outdated comments
Doy-lee Jun 5, 2025
8ac41cf
Add tmp work-around for DB swarm dump on migration to DB v1
Doy-lee Jun 5, 2025
91fa900
Infinite retries with exponential fallback
Doy-lee Jun 12, 2025
c3f97e8
Get rid of duplicate fnv hash, already present in serialize.h
Doy-lee Jun 12, 2025
be3bd39
Remove unused TEST_RETRY_INTERVAl/PERIOD constants
Doy-lee Jun 12, 2025
7b4bee5
Rename BTSerialise to Serialise and prefix SN's SerialiseResult to be…
Doy-lee Jun 12, 2025
3924d1f
Remove unused MessageTestStatus
Doy-lee Jun 12, 2025
ad10794
Move retryable requests into its own thread and restore pending swarm…
Doy-lee Jun 12, 2025
c1ee078
Revert SN data ready response, timestamp not needed
Doy-lee Jun 13, 2025
936c810
Serialise retryable requests to the DB
Doy-lee Jun 16, 2025
18c7c7a
Match comment w/ code regarding when to add retryable request on failure
Doy-lee Jun 17, 2025
0c0c5e5
Revert NEW_SWARM_MEMBER_INTERVAL to 10s
Doy-lee Jun 17, 2025
9da0460
Update the swarms serialisation to match retryable requests patterns
Doy-lee Jun 17, 2025
48c3de0
Remove unused swarm_member_check_deadline, add comments
Doy-lee Jun 17, 2025
a36b208
Simplify swarm member status states by removing ContactDetailsReady
Doy-lee Jun 17, 2025
5a67656
Fix dblocation typo -> db_location
Doy-lee Jun 17, 2025
aa5fe96
Remove RETRY_BACKOFF_COEFF from retry request on done lambda
Doy-lee Jun 17, 2025
f52aecd
Linting
Doy-lee Jun 17, 2025
9153442
Request a DB dump on data-ready handshake if DB is empty
Doy-lee Jun 19, 2025
b9f5d2d
Add 10GiB message payload to serialise test
Doy-lee Jun 23, 2025
9ac6c91
Linting
Doy-lee Jun 27, 2025
26b6b22
Reduce serialisation test to 5GiB to not OOM on CI
Doy-lee Jun 27, 2025
6fd8391
Reduce serialisation test to 100MiB to not OOM on CI
Doy-lee Jun 27, 2025
5a8f1de
Stop potential infinite bootstrap of DB if swarm ID doesn't change
Doy-lee Jul 14, 2025
2962332
Change retryable requests storage format
tewinget Feb 17, 2026
0f91aca
Add swarm space/id cache to database
tewinget Mar 3, 2026
def6b8e
remove dead code and C++-ify some C-esque code
tewinget Mar 4, 2026
6bfd954
format
tewinget Mar 4, 2026
13fde68
Do not store public outbox message if newer present
tewinget Mar 4, 2026
726f082
add limit (and order) to query
tewinget Mar 4, 2026
2bcc0c6
Add unique index on public namespace and owner
tewinget Mar 4, 2026
3aa2762
fix unused variable errors
tewinget Mar 5, 2026
e1e4be2
fix unit tests, including broken db queries
tewinget Mar 5, 2026
bb52e7e
unused parameter
tewinget Mar 5, 2026
d015af3
new tests and fixes found from them
tewinget Mar 26, 2026
118b0f1
more thorough swarm id -> space testing
tewinget Mar 26, 2026
b65fd43
reformat some values and comments
tewinget Mar 26, 2026
ba54fa0
Replace startup_version with had_swarm_state_on_open
tewinget Mar 27, 2026
f048263
Add migration test binary and historical database fixture
tewinget Mar 27, 2026
6ee099c
clang-format
tewinget Mar 27, 2026
87e6e4c
fix simple query mistakes
tewinget Apr 1, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 3 additions & 4 deletions network-tests/test_subaccount_auth.py
Original file line number Diff line number Diff line change
Expand Up @@ -336,7 +336,7 @@ def test_revoke_subaccount(omq, random_sn, sk, exclude):
f"revoked_subaccounts{ts}".encode(), encoder=Base64Encoder
).signature.decode(),
}
).encode(),
).encode()
],
).get()
assert len(r) == 1
Expand Down Expand Up @@ -408,15 +408,14 @@ def test_revoke_subaccount(omq, random_sn, sk, exclude):
f"revoked_subaccounts{ts}".encode(), encoder=Base64Encoder
).signature.decode(),
}
).encode(),
).encode()
],
).get()
assert len(r) == 1
r = json.loads(r[0])
assert len(r["revoked_subaccounts"]) == 1
assert r["revoked_subaccounts"][0] == b64(dude_token)


# But the one in the revoked-keys-allowed namespace should work:
r = omq.request_future(
conn,
Expand Down Expand Up @@ -511,7 +510,7 @@ def test_revoke_subaccount(omq, random_sn, sk, exclude):
f"revoked_subaccounts{ts}".encode(), encoder=Base64Encoder
).signature.decode(),
}
).encode(),
).encode()
],
).get()
assert len(r) == 1
Expand Down
12 changes: 12 additions & 0 deletions oxenss/common/pubkey.cpp
Original file line number Diff line number Diff line change
@@ -1,11 +1,23 @@
#include "pubkey.h"
#include "mainnet.h"
#include "oxenc/endian.h"
#include <oxenc/hex.h>
#include <charconv>
#include <cassert>

namespace oxenss {

uint64_t pubkey_to_swarm_space(const user_pubkey& pk) {
const auto bytes = pk.raw();
assert(bytes.size() == 32);

uint64_t res = 0;
for (size_t i = 0; i < bytes.size(); i += 8)
res ^= oxenc::load_big_to_host<uint64_t>(bytes.data() + i);

return res;
}

user_pubkey& user_pubkey::load(std::string_view pk) {
if (pk.size() == USER_PUBKEY_SIZE_HEX && oxenc::is_hex(pk)) {
uint8_t netid;
Expand Down
9 changes: 7 additions & 2 deletions oxenss/common/pubkey.h
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
#pragma once

#include <cstdint>
#include <string>

namespace oxenss {
Expand All @@ -13,14 +14,14 @@ class user_pubkey {
int network_ = -1;
std::string pubkey_;

user_pubkey(int network, std::string raw_pk) : network_{network}, pubkey_{std::move(raw_pk)} {}

friend class DatabaseImpl;

public:
// Default constructor; constructs an invalid pubkey
user_pubkey() = default;

user_pubkey(int network, std::string raw_pk) : network_{network}, pubkey_{std::move(raw_pk)} {}

// bool conversion: returns true if this object contains a valid pubkey
explicit operator bool() const { return !pubkey_.empty(); }

Expand Down Expand Up @@ -57,6 +58,10 @@ class user_pubkey {
std::string prefixed_raw() const;
};

/// Maps a pubkey into a 64-bit "swarm space" value; the swarm you belong to is whichever one
/// has a swarm id closest to this pubkey-derived value.
uint64_t pubkey_to_swarm_space(const user_pubkey& pk);

} // namespace oxenss

namespace std {
Expand Down
1 change: 0 additions & 1 deletion oxenss/crypto/keys.h
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,6 @@
#include <array>
#include <cstddef>
#include <functional>
#include <optional>
#include <string>
#include <string_view>

Expand Down
5 changes: 5 additions & 0 deletions oxenss/daemon/command_line.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -175,6 +175,11 @@ parse_result parse_cli_args(int argc, char* argv[]) {
"--force-start",
options.force_start,
"Ignore the initialisation ready check (primarily for debugging).");
cli.add_flag(
"--skip-bootstrap-nodes",
options.skip_bootstrap,
"Skip the contacting of bootstrap seed nodes on startup (primarily for private node "
"networks)");
cli.add_option(
"--stats-access-key",
options.stats_access_keys,
Expand Down
1 change: 1 addition & 0 deletions oxenss/daemon/command_line.h
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@ struct command_line_options {
uint16_t https_port = 22021;
uint16_t omq_quic_port = 22020;
std::string oxend_omq_rpc; // Defaults to ipc://$HOME/.oxen/[testnet/]oxend.sock
bool skip_bootstrap = false;
bool force_start = false;
bool testnet = false;
std::string log_level = "info";
Expand Down
7 changes: 6 additions & 1 deletion oxenss/daemon/oxen-storage.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -153,7 +153,12 @@ int main(int argc, char* argv[]) {
auto& oxenmq_server = *oxenmq_server_ptr;

snode::ServiceNode service_node{
l_keys, me, oxenmq_server, options.data_dir, options.force_start};
l_keys,
me,
oxenmq_server,
options.data_dir,
options.force_start,
options.skip_bootstrap};

rpc::RequestHandler request_handler{service_node, channel_encryption, ed_keys.sec};

Expand Down
10 changes: 5 additions & 5 deletions oxenss/rpc/client_rpc_endpoints.h
Original file line number Diff line number Diff line change
Expand Up @@ -541,9 +541,9 @@ struct delete_before final : recursive {
};

/// Updates (shortens) the expiry of all stored messages, and broadcasts the update request to all
/// other swarm members. Note that this will not extend existing expiries, it will only shorten the
/// expiry of any messages that have expiries after the requested value. (To extend expiries of one
/// or more individual messages use the `expire` endpoint).
/// other swarm members. Note that this will not extend existing expiries, it will only shorten the
/// expiry of any messages that have expiries after the requested value. (To extend expiries of one
/// or more individual messages use the `expire_msgs` endpoint).
///
/// Takes parameters of:
/// - pubkey -- the pubkey whose messages shall have their expiries reduced, in hex (66) or bytes
Expand Down Expand Up @@ -625,8 +625,8 @@ struct expire_all final : recursive {
/// ("expire" || ShortenOrExtend || expiry || messages[0] || ... || messages[N])
/// where `expiry` is the expiry timestamp expressed as a string, for a single expiry, or the
/// expiries concatenated together (expiry[0] || expiry[1] || ...) for multiple expiries.
/// `ShortenOrExtend` is string "shorten" if the shorten option is given (and true), "extend" if
/// `extend` is true, and empty otherwise. The signature must be base64 encoded (json) or bytes
/// `ShortenOrExtend` is the string "shorten" if the shorten option is given (and true), "extend"
/// if `extend` is true, and empty otherwise. The signature must be base64 encoded (json) or bytes
/// (bt).
///
/// Returns dict of:
Expand Down
Loading