Improved sync speed by resolving the N+1 query issues.
Solves #1402 and Solves #1453
With this change there is just one query done to retreive all the
important data, and matching is done in-code/memory.
With a very large database the sync time went down about 3 times.
Also updated misc crates and Github Actions versions.
The armv6 builds need a specific location for the libatomic.a file.
This commit fixes that by adding a RUSTFLAGS argument for this.
Also removed the `link-arg=-s` since this is now already done during via the release profile
And removed the CFLAGS for armv7, this is already fixed by default in the blackdex/rust-musl images.
- Removed all `thread::sleep` and use `tokio::time::sleep` now.
This solves an issue with updating to Bullseye ( Resolves #1998 )
- Updated all Debian images to Bullseye
- Added MiMalloc feature and enabled it by default for Alpine based images
This increases performance for the Alpine images because the default
memory allocator for MUSL based binaries isn't that fast
- Updated `dotenv` to `dotenvy` a maintained and updated fork
- Fixed an issue with a newer jslib (not fully released yet)
That version uses a different endpoint for `prelogin` Resolves #2378 )
Favicon:
- Replaced HTML tokenizer, much faster now.
- Caching the domain blacklist function.
- Almost all functions are async now.
- Fixed bug on minimizing data to parse
- Changed maximum icon download size to 5MB to match Bitwarden
- Added `apple-touch-icon.png` as a second fallback besides `favicon.ico`
SMTP:
- Deprecated SMTP_SSL and SMTP_EXPLICIT_TLS, replaced with SMTP_SECURITY
Misc:
- Fixed issue when `resolv.conf` contains errors and trust-dns panics (Fixes #2283)
- Updated Javscript and CSS files for admin interface
- Fixed an issue with the /admin interface which did not cleared the login cookie correctly
- Prevent websocket notifications during org import, this caused a lot of traffic, and slowed down the import.
This is also the same as Bitwarden which does not trigger this refresh via websockets.
Rust:
- Updated to use v1.59
- Use the new `strip` option and enabled to strip `debuginfo`
- Enabled `lto` with `thin`
- Removed the strip RUN from the alpine armv7, this is now done automatically
- Changed nightly to stable in Dockerfile and Workflow
- Updated Dockerfile to use stable and updated ENV's
- Removed 0.0.0.0 as default addr it now uses ROCKET_ADDRESS or the default
- Updated Github Workflow actions to the latest versions
- Updated Hadolint version
- Re-orderd the Cargo.toml file a bit and put libs together which are linked
- Updated some libs
- Updated .dockerignore file
- Using my own rust-musl build containers we now support all database
types for both Debian and Alpine.
- Added new Alpine containers for armv6 and arm64/aarch64
- The Debian builds can also be done wihout dpkg magic stuff, probably
some fixes in Rust regarding linking (Or maybe OpenSSL or Diesel), in
any case, it works now without hacking dpkg and apt.
- Updated toolchain and crates
This fixes #1998 where with some checking it seems Bullseye has some
issues with the glibc sleep call. It returns a SIGILL.
The glibc on Buster doesn't seem to have this issue, so revert back for
now until a fix has been released.
- Split Debian and Alpine into different build matrix
This starts building both Debian and Alpine based images at the same time
- Make use of Docker BuildKit, which improves speed also.
- Use BuildKit caching for Rust Cargo across docker images.
This prevents downloading the same crates multiple times.
- Use Github Actions Services to start a docker registry, starting it
via the build script sometimes caused issues.
- Updated the Build workflow to use Ubuntu 20.04 which is more close to
the Bullseye Debian release regarding package versions.
Updated several dependencies and switch to different totp library.
- Switch oath with totp-lite
oauth hasn't been updated in a long while and some dependencies could not be updated any more
It now also validates a preseeding 0, as the previous library returned an int instead of a str which stripped a leading 0
- Updated rust to the current latest nightly (including build image)
- Updated bootstrap css and js
- Updated hadolint to latest version
- Updated default rust image from v1.53 to v1.54
- Updated new nightly build/clippy messages
- Removed azure-pipelines
- Updated gh-actions to run `cargo test` per db feature
- Fail on warnings by adding `RUSTFLAGS` env
- Updated Dockerfile to fix some new hadolint warnings
- Fixed bug when web-vault is disabled.
- Updated sql-server version check to be simpler thx to @weiznich ( https://github.com/dani-garcia/bitwarden_rs/pull/1548#discussion_r604767196 )
- Use `VACUUM INTO` to create a SQLite backup instead of using the external sqlite3 application.
- This also removes the dependancy of having the sqlite3 packages installed on the final image unnecessary, and thus removed it.
- Updated backup filename to also have the current time.
- Add specific bitwarden_rs web-vault version check (to match letter patched versions)
Will work when https://github.com/dani-garcia/bw_web_builds/pull/33 is build (But still works without it also).
Some small changes in general:
- Moved the SQL Version check struct into the function.
- Updated hadolint to 2.0.0
- Fixed hadolint 2.0.0 warnings
- Updated github workflows
- Added .editorconfig for some general shared editor settings.
The bitwarden_rs code is still cross-compiled exactly as before, but Docker
Buildx is used to rewrite the resulting Docker images with correct platform
metadata (reflecting the target platform instead of the build platform).
Buildx also now handles building and pushing the multi-arch manifest lists.
This is useful for making local customizations upon container start. To use
this feature, mount a script into the container as `/etc/bitwarden_rs.sh`
and/or a directory of scripts as `/etc/bitwarden_rs.d`. In the latter case,
only files with an `.sh` extension are sourced, so files with other
extensions (e.g., data/config files) can reside in the same dir.
Note that the init scripts are run each time the container starts (not just
the first time), so these scripts should be idempotent.
* Switch healthcheck interval/timeout from 30s/3s to 60s/10s.
30s interval is arguably overkill, and 3s timeout is definitely too short
for lower end machines.
* Use HEALTHCHECK CMD exec form to avoid superfluous `sh` invocations.
* Add `--silent --show-error` flags to curl call to avoid progress meter being
shown in healthcheck logs.
The muslrust images seem to have a workdir of /volume as opposed to / in the
others so doing cargo new like this would create the folder in /volume/app.
This is done to enable backup functionality in the admin interface while
we're waiting for the libsqlite-sys 0.17 to bubble up in the upstream
dependencies. Then we can start using `VACUUM INTO`
This also extends the check for the sqlite binary to also try `sqlite3`
as this is the name of the binary in baseimage distributions we use.
This changes the healthcheck to use `sh` instead of bash, that is absent
from some image versions. (like alpine)
It also removes `*mariadb*` packages from runtime image of sqlite images
as these shouldn't be required.
