CHANGES ------- The major number changes for such things as code rewrites, changes in syntax, and/or dramatic changes in functionality. The minor number changes for corrections, enhancements, etc. made to the code. There is no set schedule for releases. --TODO-- --0.20-- + Requires LibSnert 1.63 + Added siq-query option for when libmilter is built with -DSMFI_VERSION=4 so that SIQ queries can be delayed until the DATA command. Requested by April Lorenzen. + Recipient B/W listing possible with siq-query=DATA. + Added cache-ttl to impose an upper TTL limit for the TTL given in the SIQ response. + access-db now supports socket-map, flat-file, and SQLite3 lookups in addition to Berkeley DB. The SQLite3 behaviour can be disabled via LibSnert's ./configure script. ! Corrected documentation error concerning two cache-type options when it should have been cache-file and cache-type. ! cachePut() and cacheGet() now compact/uncompact the SIQ structure before caching it in order to avoid caching the unused bytes of the text and extra fields. ! atExitCleanUp(): added NULL guard around cache cleanup code in case its the cache was not initialised. --0.19-- !! License 1.4 which provides additional details with regards to possible license controls used, the possibility of "phone home code" and its general contents, and related privacy statement. + Add check for filterOpen() got NULL socket address pointer passed to filterOpen() which inidicates an unsupported address type. The connection is simply accepted to bypass the milter. --0.18-- + Requires LibSnert 1.61 ! Fix caching support broken in 0.14. !! Fixed UDP packet length bug. Reported by no one. !! Complete replacement of option specification and no backwards. compatibility. The newer format allows for better integration of previously undocumented libsnert options and for the inclusion of future enhancements. ! filterClose(): remove reporting of "null workspace error", which can occur when a client connection is rejected before or by filterOpen(). ! Minor corrections for gcc -Wall warnings. --0.17-- ! Reverted back to libmilter's default (7210s) timeout. This should silence many of the reported sendmail to milter timeout errors and/or broken connections caused by the milter timing out sooner than sendmail does with the current SMTP client. The sendmail defaults for each SMTP command and receiving the message content is 1h each and so libmilter allows for 2h 10s. But in theory you could be DoS for between 5 and 104 hours per connection. If you want to change this then set in sendmail.mc: define(`confTO_DATABLOCK', `30m') define(`confTO_COMMAND', `5m') --0.16-- + Requires LibSnert 1.57 + Added support for milter-length-auth:auth_authen RHS milter-length-auth: RHS --0.15-- + Requires LibSnert 1.56 ! Switched over to using socket2 API. ! Changed install.sh.in to create /var/run/milter/ directory owned by milter:milter and change the default PID_FILE and SOCKET_FILE paths. This provides better privelage/security separation since the milters run as milter:milter. --0.14-- + Requires LibSnert 1.55 ! Updated documentation to reflect libsnert changes for B/W listing with respect to new NEXT action. --0.13-- + Requires LibSnert 1.52 + Supports Internet draft-irtf-asrg-iar-howe-siq-02.txt. --0.12-- + Requires LibSnert 1.49 ! X-SIQ-Score header renamed in favour of SIQ-Score header. The header is prefixed with a distinct namespace and considered sufficiently unique so as not to conflict with other headers. There was some discussion as to the use of X- headers and the difficultly in their being renamed once widely adopted. Better now than later. - Moved siq API & CLI into libsnert. --0.11-- ! install.sh: fix use of id -u, which doesn't work on SunOS ! install.sh: fix use of bang (!) logical-NOT operator, which is not available in real Bourne shells or csh. ! install.sh: SunOS grep does not have a -q option ! install.sh: fixed "if which command ; then" tests with a function. which(1) always returns success (0) on SunOS. ! install.sh: SunOS has two different install(1) tools with different options (-o vs. -u) to set the file owner. + Added siq CLI. To build: make -f extra.mk build --0.10-- + Requires LibSnert 1.41 + Supports Internet draft draft-irtf-asrg-iar-howe-siq-01.txt. ! Fixed configure.in script to better handle Berkeley DB library and header searches. All these cases should now work. ./configure ./configure --with-db ./configure --with-db=/usr/local/BerkeleyDB-4.2 ./configure --without-db ! milter crashed when no SIQ servers specified. Reported by Philipp Baer. ! Fixed time stamp variables to use the time_t instead of assuming a long. + configure.in: now expects a user and group milter (instead of smmsp/smmsp). The old behaviour can be obtained with: ./configure --enable-run-user=smmsp --enable-run-group=smmsp ! Fixed the start-up script to unalias shell function names before defining the function. Reported by Francisco of Blackant dot Net. ! Encapsulated announce/pass/fail output the startup-script to support different OS favours; some like pretty output, some like it simple. - configure.in: Removed gcc option -fvolatile, which is no longer supported in as of gcc 3.4. ! Fix bug where random IPv6 queries from an IPv4 only host were being made. The problem occured when an SMTP connection passed more than one message per connection and we'd try to reuse the same IP from the first query. The query packet contained the only copy of the IP, but this same buffer was used for the response packet, which would destroy that copy. Solution, have a distinct query and response packet buffers. + The milter now starts as a background process and can kill a previous instance of itself using the -x option. The -X option is undocumented, but starts the application as a foreground process for the purpose of debugging. + Added support for a /etc/mail/milter-name.cf file that can contain command-line options and an optional socket argument. ! The socket argument is now optional. ! The above three changes allow for a simplified startup script template for use by all my milters. !! Updated LICENSE.TXT. --0.9-- + Requires LibSnert 1.40 ! Renamed DataInitFromBytes() -> DataInitWithBytes() to remain consistent with my naming/action conventions. A *FromType is a copy from source and a *WithType is an assignment and/or passing of responsiblity of source. --0.8-- ! Inverted test for helohost == NULL to != NULL for HL/ML bit support requried by OutboundIndex.org --0.7-- + Bumped version to clearly indicate a new version after some people got hold of pre-release work in progress. --0.6-- + Requires LibSnert 1.39 + Bumped version to clearly indicate a new version after some people got hold of pre-release work in progress. ! Better cache debug logging. ! Fixed bug where response text was not being saved in the cache. ! Also fixed a problem with casts setting the cached score to zero. ! flatfile cache type should now work correctly. + Added -z option to select cache type and file name, replacing compile time options. ! Fixed FreeBSD cache problem with smfSetProcessOwner() before any files (specifically the cache) are opened. + contrib/milter-siq-cachedump.pl script added. ! Fixed the configuration file to recognise OpenBSD. Reported by Francisco of Blackant dot Net. ! Fixed the start-up script to unalias shell function names before defining the function. Reported by Francisco of Blackant dot Net. ! White listed connections were not being skipped. --0.5-- + Requires LibSnert 1.37 ! UDP query format updated for draft-ietf-asrg.iar-howe-siq-00.txt. ! atExitCleanUp() now calls smdbClose() which mutex protects the call to db->close(). For the access database this is not so important, but it clearer code. + atExitCleanUp() mutex protects the call to cache->sync() and cache->close() --0.4-- + Requires LibSnert 1.36 ! Fix makefile ownership target to assert correct state directory permissions. + Set working directory to the state directory, where is can write a core file in the event of crash. Remember in FreeBSD to set sysctl kern.sugid_coredump=1 ! nullWorkSpace() now takes a `where' string to report in what hook it failed. ! Replaced hostname and hostaddr variables by ${if_name} and ${if_addr} macros. ! Replaced some common code shared by all my milters. See libsnert Sendmail Filter API (smf). ! Standardised the -v option across all my milters. Also support textual bit names. ! Removed requirement for ${client_name} and ${client_addr} macros. Now use parameters given to filterOpen(). - Removed Spam: OK tag variant. ! Increase default for -t to 1800s (30m) to account for a long SMTP connections with multiple messages. ! Enabled by default From: and To: support. + Updated ./configure script to support --enable-cache. + Add VERSION.TXT.IN to configuration for use by sendmail.shtml. ! Should work correctly now within FreeBSD jails. ! Updated for draft-howe-siq-00.txt ! Fixed exponential backoff algorithm + Implemented cache with default 30m TTL. ! Changed default value for -T from 75 to 50. ! Modified license. + Added special support for HL and ML bits used by Outbound Server Authentication Index. Rationale:- Quickly: the combination of the HL and ML flags set true (1) indicates that the MAIL FROM: is less likely to be forged, because of a positive relationship between the connecting client IP, the resolved HELO argument, and MAIL FROM: domain. A SIQ server may optionally use the information represented by these flags. The problem space addressed by the HL and ML bits: Assumption 1. The reputation result categories possible for an IP/domain pair are limited to these three: - facts about the IP - relationship, if any, between the IP and domain - facts about the domain Assumption 2. "Facts about the domain" should be ignored unless a relationship of some type can be established between the IP and domain. Assumption 3. A "relationship between IP and domain" could be inferred by many different factors, depending on the type of business, such as: - SPF or other self-declared designated sender relationship - Shared name servers - Same owner organization - Same small swipped block - Indications that the admin who controls DNS for the domain, has pointed some FQDN containing that domain, to the IP. (He or she is clearly AWARE OF that IP performing some function for that domain.) The authors ask that the reader bear in mind for the next paragraph that we are not looking for reasons to REJECT email - rather we are looking for reasons to ACCEPT email. The bane of many anti-spam measures which correctly identify senders of spam is that they also incorrectly label the "sub-optimally configured" legitimate sender. For example, the connecting IP used by many spammers does not have a PTR record, or has a forged PTR record, or has forward and reverse DNS which do not match. Unfortunately, the same is true for many "sub- optimally configured" legitimate senders. In some cases there may be almost nothing (pre-DATA) to differentiate the "sub-optimally configured" legitimate sender from a spammer - other than the legitimate sender's domain name reputation such as: - identifiability (Is the domain owner hiding or can he be easily and directly located for legal action or repercusssions if activities in email are unlawful?) - longevity (How long has their domain been registered, appeared in the email system, company been recognized officially, ISP / name servers etc.) - stability (How stable and consistent is his use of servers for this domain and others he owns?) Illicit senders tend toward instability, hiding, moving and appearing from constantly changing "fresh" combinations. Being able to use the legitimate sender's domain name in scoring is therefore very useful. But, given the assumptions outlined above, the domain name cannot be used in scoring unless some relationship can be inferred to the IP address of the connecting server. The HELO/IP/MAIL test (described next) establishes a relationship between the connecting IP and MAIL FROM domain name in a substantial number of cases for this very common configuration of legitimate business senders who run an in-house mail server. It also allows the hobbyist user with his own clean domain, but no reverse delegation, a way to "designate" his IP. HL Equals true (1) if the HELO argument is a FQDN that resolves to the connecting client IP. ML Equals true (1) if the MAIL FROM base domain is the same as the HELO argument base domain. The condition ML=1 alone can be created by any spammer, forging the HELO to equal the MAIL FROM domain he is choosing to forge, and does not own. However, to make the HELO argument a FQDN that resolves to the connecting client IP - proves control of DNS for that domain. Therefore, if the MAIL FROM base domain matches the HELO argument base domain as described above - a line of control and relationship is inferred between the MAIL FROM domain and connecting IP. If HL and ML are both true, it follows that the DNS admininstrator for the MAIL FROM domain is designating this IP as performing some function for this domain. This is either an authorized outbound email server for that domain, or if this IP is not authorized, other methods can be used to declare or detect that it is specifically NOT authorized as a mail server, or the third possibility - the administrators of this network are not enforcing security measures on their network, thus allowing unauthorized use of an IP or server on their network. As with nearly all anti-spam or whitelisting measures - HELO/IP/MAIL is not intended to unconditionally whitelist all messages which pass this test. Complementary tests would reveal and wash out this score in obvious cases such as a home cable user zombie IP, which has matching forward and reverse DNS as well as using the same cable ISP domain in the envelope-from. Right-hand-side blacklists would wipe out the benefit of the HELO/IP/MAIL contribution to the score, if a spammer used the tuple with his own domain and either his own, or a hijacked server IP. To re-iterate, if HELO/IP/MAIL test fails - there must be NO consequence or impact on the score that leads toward rejection. This test may only aid in the scoring towards accepting or whitelisting a message. SIQ protocol query clients are not required to send HL and ML bits and SIQ protocol servers are not required to utilize HL and ML bits. --0.3-- + Requires LibSnert 1.35 for major smdb() update to solve threading issues and a possible memory corruption problem ! Renamed -d, -r, and -t to -D, -R, and -T. + Added -t option to set the sendmail/milter socket timeout + Add an instanceCount for independent tagging of log message separate from the queue id. ! vSetReply() not checks for an MI_FAILURE from smfi_setreply(). --0.2-- + Requires LibSnert 1.34. + Added -S option. --0.1-- + Requires LibSnert 1.33. + Added contrib/milter-lives.sh cron script check that my milters remain running.