- We need to add some feature to enable smtp debugging again. See: https://github.com/lettre/lettre/pull/584
- Upstream added the fallback icon again, probably because of caching ;). See: https://github.com/bitwarden/server/pull/1149
- Enabled gzip and brotli compression support with reqwest. Some sites seem to force this, or assume that because of the User-Agent string it is supported. This caused some failed icons.
Fixes #1540
- Changed the user-agent, which caused at least one site to stall the
connection (Same happens on icons.bitwarden.com)
- Added default_header creation to the lazy static CLIENT
- Added referer passing, which is checked by some sites
- Some small other changes
- Added more checks to prevent panics (Removed unwrap)
- Try do download from base domain or add www when the provided domain
fails
- Added some more domain validation checks to prevent errors
- Added the ICON_BLACKLIST_REGEX to a Lazy Static HashMap which
speeds-up the checks!
- Validate the Regex before starting/config change.
- Some cleanups
- Disabled some noisy debugging from 2 crates.
Some sites are using base64 encoded inline images for favicons.
This will try to match those with some sane checks and return that.
These icons will have lower prio then the icons with a normal URL.
Now creates icon cache directory at startup.
And it also creates the directory if it went missing during runtime.
Also modified the icon_save/mark_negcache to be one.
When the icon_cache directory doesn't exists yet, and the first icon
catched is a miss this .miss file was not able to be created since the
directory was only created during a valid icon download.
Some sites use XSRF Tokens, or other Tokens to verify a subseqense
response. The cookies which are sent during the page request are now
used when downloading the favicon.
A site which uses this is mijn.ing.nl.
- Using url from reqwest to fix href, this fixes:
+ "//domain.com/icon.png"
+ "relative/path/to/icon.png"
+ "/absolute/path/to/icon.png"
- Removed fix_href function
- Some variable changes
