CanIt-Domain-PRO Administration Guide for Version 10.2.9 AppRiver, LLC 11 December 2020 2
CanIt-Domain-PRO — AppRiver, LLC Contents
1 Introduction 19 1.1 Principles of Operation...... 19 1.2 Handling False-Positives...... 19 1.2.1 Spam-Control Delegation...... 20 1.3 Organization of this Manual...... 20 1.4 Definitions...... 21
2 Operation 27 2.1 Principles of Operation...... 27 2.2 Interaction between Allow Rules and Block Rules...... 28 2.2.1 RCPT TO: Actions...... 28 2.2.2 Post-DATA Actions...... 30 2.3 Streaming...... 32 2.4 How Addresses are Streamed...... 32 2.5 How Streaming Methods are Chosen...... 33 2.6 Status of Messages...... 35 2.6.1 Secondary MX Relays...... 36 2.7 The Database...... 36 2.8 Remailing Messages...... 37
3 Realms 39 3.1 Introduction to Realms...... 39 3.2 Realm Names...... 40 3.2.1 The base Realm...... 40 3.3 Creating Realms...... 40 3.4 Realm Mappings...... 41 3.5 Determining the Realm...... 42
CanIt-Domain-PRO — AppRiver, LLC 3 4 CONTENTS
3.5.1 Mapping a Domain to a Realm...... 42 3.5.2 Mapping an Address to a Realm...... 42 3.5.3 Mapping a Login Name to a Realm...... 43 3.6 Realm Expiry...... 43 3.6.1 Suspending Service to a Realm...... 43 3.7 Realm Hierarchy...... 43 3.8 Realm Custom Fields...... 44
4 Streams 47 4.1 Introduction to Streams...... 47 4.2 Realms...... 47 4.3 The Definition of a Stream...... 47 4.4 Users and E-Mail Addresses...... 47 4.5 Mapping...... 50 4.6 The Home Stream...... 50 4.7 The “default” Stream...... 51
5 CanIt-Domain-PRO Setup 53 5.1 Accessing The Web Interface...... 53 5.1.1 License Key Screen...... 53 5.1.2 Login Screen...... 54 5.2 The Setup Menu...... 55 5.3 Wizards...... 56 5.3.1 Basic Setup Wizard...... 56 5.3.2 RPTN Setup Wizard...... 56 5.3.3 Dictionary Attack Detection Wizard...... 56 5.4 Verification Servers...... 57 5.4.1 Wildcard Verification Server...... 59 5.4.2 SRS and Verification Servers...... 60 5.5 Mail Routing...... 60 5.5.1 Outbound Relaying...... 62 5.5.2 Outbound Relaying for Select Domains...... 62 5.6 Cluster Management...... 63 5.6.1 Bandwidth Optimization for Copying Files...... 64 5.6.2 Altering Services on a Cluster Member...... 64 5.6.3 Renaming of Cluster Members...... 65
CanIt-Domain-PRO — AppRiver, LLC CONTENTS 5
5.7 Known Networks...... 65 5.7.1 Associating Domains with Known Networks...... 68 5.7.2 Overlapping Networks...... 69 5.7.3 The SMTP-AUTH Pseudo-Network...... 69 5.8 Rate-Limiting Outbound Mail...... 69 5.8.1 Rate-Limiting by IP Address...... 71 5.8.2 Fine-Grained Rate-Limiting Rules...... 71 5.8.3 Notes about Rate-Limiting Rules...... 73 5.9 Features...... 74 5.9.1 Direct Queue Injection...... 74 5.10 System Check...... 75 5.11 Templates...... 76 5.12 Theme Customization and Branding...... 79 5.12.1 Creating or Editing a Customization...... 80 5.12.2 Emergency Recovery from Bad Theme Customization...... 81 5.13 HTTPS...... 81 5.14 The Domain Mapping Table...... 81 5.15 The Address Mapping Table...... 83 5.15.1 Wild-Card Entries...... 84 5.16 The default Stream...... 85 5.17 Mapping Scenarios...... 85 5.17.1 Central Scanning with Opt-Out...... 85 5.17.2 Single Domain...... 86 5.17.3 Single Domain with Aliases and Mailing Lists...... 86 5.18 Pausing Delivery to Selected Domains...... 86 5.18.1 Pausing Delivery...... 86 5.18.2 Resuming Delivery...... 87 5.19 The Domain Overview Page...... 87 5.20 Autotask® Integration...... 88 5.20.1 Preparing Autotask...... 88 5.20.2 Preparing CanIt-Domain-PRO...... 90 5.20.3 Testing the Autotask Integration Settings...... 93 5.20.4 Autotask Settings and Inheritance...... 95 5.21 ConnectWise® Integration...... 95 5.21.1 Preparing ConnectWise...... 95
CanIt-Domain-PRO — AppRiver, LLC 6 CONTENTS
5.21.2 Preparing CanIt-Domain-PRO...... 103
6 CanIt-Domain-PRO Administration 105 6.1 Global Settings...... 105 6.2 SRS (Sender Rewriting Scheme)...... 109 6.3 Real-Time DNS Blocklists...... 110 6.3.1 Entering the Master List of DNS RBLs...... 110 6.3.2 combined.bl.rptn.ca...... 111 6.4 Phishing URLs...... 112 6.4.1 Malicious URL Votes...... 112 6.4.2 Known Phishing URLs...... 114 6.4.3 Delaying Messages because of local Phishing Votes...... 115 6.5 Users...... 116 6.5.1 User Privileges...... 117 6.5.2 Adding a User...... 117 6.5.3 Editing a User...... 118 6.5.4 Deleting a User...... 119 6.5.5 Granting Access to Streams...... 119 6.5.6 Switching Users...... 120 6.6 Permitting Users to Opt In...... 121 6.7 Groups...... 122 6.7.1 Creating, Deleting and Editing Groups...... 122 6.8 Viewing Active Streams...... 123 6.8.1 Definition of an Active Stream...... 124 6.8.2 The Active Stream Display...... 124 6.8.3 Deleting a Stream...... 125 6.9 Filtering Outbound Mail...... 125 6.9.1 DKIM-Signing Outbound Mail...... 125 6.10 Copying Rules from One Stream to Another...... 129 6.11 Secondary MX Hosts...... 130 6.12 Avoiding Backscatter...... 131 6.13 Test Plugins...... 131 6.13.1 The PhishingAddress Plugin...... 132 6.13.2 The PhishingURL Plugin...... 132 6.13.3 The OfficeMacros Plugin...... 132
CanIt-Domain-PRO — AppRiver, LLC CONTENTS 7
6.13.4 The OfficeMacro* Open Plugins...... 132 6.13.5 The Shortener404 Plugin...... 133 6.13.6 The NewlySeenDomain Plugin...... 133 6.14 Emergency Blocking of Delivery Status Notifications...... 133 6.15 Removing All Rules and Settings from a Stream...... 134 6.16 Provisioning Information...... 135 6.16.1 Computer-Readable Provisioning Information...... 136
7 External Authentication 137 7.1 Introduction...... 137 7.2 User Lookups...... 137 7.2.1 IMAP and POP3 Authentication...... 139 7.2.2 LDAP Authentication and Streaming...... 141 7.2.3 Azure Active Directory Streaming...... 145 7.2.4 Program Authentication and Streaming...... 153 7.2.5 Program Authentication (Legacy Method)...... 157 7.2.6 The account-info Script...... 157 7.2.7 The Rewrite User Lookup...... 157 7.3 Authentication Mappings...... 158 7.4 Viewing Cached Logins...... 159
8 Bayesian Filtering 161 8.1 Introduction to Bayesian Filtering...... 161 8.2 Unauthenticated Voting...... 161 8.3 The Bayes Journal...... 162 8.4 Site-Wide and Realm-Wide Bayes Training...... 162 8.5 RPTN...... 162 8.6 Ruleset and Geolocation Data Updates...... 163
9 Permissions 165 9.1 Introduction...... 165 9.2 Stream Permissions...... 165 9.3 Determining Permissions...... 166 9.4 Granting Permissions...... 167 9.4.1 Granting Stream Permissions...... 167 9.4.2 Granting User Permissions...... 169
CanIt-Domain-PRO — AppRiver, LLC 8 CONTENTS
9.5 Permission Grantability...... 171 9.5.1 Grantability Algorithm...... 172
10 Streams, Inheritance and the Simple GUI 173 10.1 Simplification...... 173 10.2 Stream Inheritance...... 173 10.3 Special Streams...... 175 10.3.1 Final Streams...... 175 10.3.2 Creating Special Streams...... 175 10.3.3 Deleting Special Streams...... 176 10.4 The Simplified GUI...... 176 10.5 Inheritance from Non-Final Streams...... 177 10.6 Inheritance from Opted-Out Streams...... 177
11 Periodic Reports 179 11.1 Introduction...... 179 11.1.1 Periodic Reports...... 179 11.1.2 Charts...... 179 11.2 Creating Charts...... 181 11.3 Creating Periodic Reports...... 181 11.4 Editing Periodic Reports...... 182 11.5 Running a Report on Demand...... 183
12 Locked Addresses 185 12.1 Introduction to Locked Addresses...... 185 12.2 Preparing to use Locked Addresses...... 185 12.2.1 Create a new domain...... 185 12.2.2 Configure mail for the new domain...... 185 12.2.3 Inform CanIt-Domain-PRO about the locked address domain...... 186 12.2.4 Associate each login name with an e-mail address...... 186
13 Attachment Handling 187 13.1 General Filename and MIME Type Rules...... 187 13.2 Delaying Attachments...... 187 13.2.1 Configuring the Time Delay...... 187 13.2.2 Creating Delay Rules...... 187 13.2.3 How It Works...... 188
CanIt-Domain-PRO — AppRiver, LLC CONTENTS 9
13.3 Stripping Attachments...... 189 13.3.1 Approving the Release of Stripped Attachments...... 190
14 URL Proxying 191 14.1 Configuring URL Proxying...... 192 14.2 Proxying Known Phishing URLs...... 193 14.2.1 Known Phishing Test Point...... 193
15 SMTP Server Testing 195 15.1 An SMTP Primer...... 195 15.2 Testing an SMTP Server...... 197 15.3 SMTP Test Results...... 198
16 CanIt Storage Manager 201 16.1 Storage Manager Concepts...... 201 16.1.1 Principles of Operation...... 202 16.2 Configuring the Storage Manager...... 203 16.2.1 Enabling the Storage Manager...... 203 16.2.2 The Configuration Wizard...... 203 16.2.3 Local Configuration...... 205 16.2.4 Starting the Storage Manager...... 205 16.2.5 Data Stored in the Storage Manager...... 206 16.3 Backup Considerations...... 206 16.4 Running multiple Storage Managers...... 206 16.5 ps Output...... 207
17 Searching Logs 209 17.1 Introduction...... 209 17.2 Log Basics...... 209 17.3 Searching the Logs...... 210 17.3.1 Performing a Search...... 210 17.3.2 Fields...... 211 17.3.3 Creating a Log Search Query...... 213 17.4 Saving Log Searches...... 213 17.4.1 Managing Saved Log Searches...... 213 17.5 Log Search Results...... 214 17.5.1 Detailed Results...... 215
CanIt-Domain-PRO — AppRiver, LLC 10 CONTENTS
17.5.2 Downloading Log Lines...... 215 17.6 Forwarding Logs...... 216 17.6.1 Enabling Log-Forwarding...... 216 17.6.2 Configuring Log-Forwarding...... 216
18 Tips 219 18.1 Greylisting...... 219 18.2 Don’t Trust Sender Addresses...... 220 18.3 Don’t Trust Sender Domains...... 220 18.4 You May Trust Relay Hosts...... 220 18.5 Custom Rules...... 221 18.5.1 General Recommendations...... 221 18.5.2 Things to avoid...... 221 18.6 Group High-Scoring Messages Together...... 221 18.7 AppRiver Best-Practices...... 222 18.8 General Anti-Spam Tips...... 222 18.8.1 Use Receive-Only Addresses on your Web Site...... 222 18.8.2 Do Not Reply to Spam...... 222
19 Security 223 19.1 Don’t Run as Root...... 223 19.2 Ownership and Permissions...... 223 19.3 SSH...... 224 19.4 PostgreSQL Security...... 224 19.5 PHP Security...... 224 19.6 Network Security...... 224 19.7 Backups...... 225
A The Domain Configuration Wizard 227 A.1 Introduction...... 227 A.2 Entering the Domain Name...... 227 A.3 Picking a Realm...... 227 A.4 Configuring Streaming...... 228 A.5 Configuring Authentication...... 229 A.6 Configuring Routing and Verification...... 230 A.7 Summary...... 231
CanIt-Domain-PRO — AppRiver, LLC CONTENTS 11
B Release Notes 233
C A Testing Topology for CanIt-Domain-PRO 331 C.1 Introduction...... 331 C.2 Assumptions...... 331 C.3 Network Setup...... 331 C.4 Build the CanIt-Domain-PRO Server...... 332 C.5 Configure the CanIt-Domain-PRO Server to Relay Mail...... 332 C.5.1 Enable Relaying...... 333 C.5.2 Configure Forwarding Relays...... 333 C.5.3 Rebuild Sendmail Databases...... 333 C.6 Route Test Mail...... 333 C.6.1 Direct Injection...... 334 C.6.2 Create a Test Subdomain...... 334 C.7 Route Real Mail...... 334 C.8 Outgoing Mail...... 335
D CanIt-Domain-PRO Architecture 337 D.1 Introduction...... 337 D.2 CanIt-Domain-PRO Architecture...... 338 D.3 Starting and Stopping CanIt-Domain-PRO...... 339 D.4 Static Configuration Files...... 340 D.4.1 Database Settings...... 340 D.4.2 Cron Settings...... 340 D.4.3 MIMEDefang Settings...... 341 D.4.4 Filter Settings...... 343 D.4.5 Ticker Settings...... 344 D.4.6 Cluster Communication Settings...... 345 D.4.7 Storage Manager Settings...... 345 D.4.8 Maintenance Notification...... 346 D.5 Tuning CanIt-Domain-PRO...... 346 D.5.1 Memory...... 347 D.5.2 Disk...... 347 D.5.3 Solaris-Specific tmpfs Note...... 347 D.5.4 CPU...... 347 D.5.5 Sendmail...... 347
CanIt-Domain-PRO — AppRiver, LLC 12 CONTENTS
D.6 Dealing with Overload...... 348 D.6.1 Tune CanIt-Domain-PRO and Sendmail...... 348 D.6.2 Network Architecture...... 348
E CanIt-Domain-PRO HOWTOS 349 E.1 Restoring a Database from a Dump...... 349 E.2 Firewall Settings...... 350 E.2.1 Firewall Rules: External Hosts...... 350 E.2.2 Firewall Rules: Internal Hosts...... 350 E.2.3 Firewall Rules: Intra-Cluster Hosts...... 351 E.3 Running Something after the Nightly Cron Job Completes...... 351 E.4 Hooks...... 352 E.5 Migrating CanIt-Domain-PRO to a Different Machine...... 352 E.5.1 CanIt-Domain-PRO Clusters...... 353 E.5.2 Storage Manager...... 353 E.5.3 Migration Procedure...... 353 E.6 Cloning a CanIt-Domain-PRO Machine...... 356
F Using CanIt-Domain-PRO with memcached 357 F.1 Introduction...... 357 F.2 Using memcached...... 357 F.2.1 Installing memcached...... 357 F.2.2 Configuring memcached...... 357 F.2.3 Single vs. Multiple Caches...... 358 F.2.4 Configuring CanIt-Domain-PRO to use memcached...... 358 F.3 What is Cached...... 359
G Using CanIt-Domain-PRO with PgBouncer 361 G.1 Introduction...... 361 G.2 Installation...... 361 G.3 Configuration...... 361 G.3.1 Configuring userlist.txt ...... 362 G.3.2 Configuring pgbouncer.ini ...... 362 G.3.3 Configuring CanIt-Domain-PRO to use PgBouncer...... 362
H CanIt-Domain-PRO Logging 365 H.1 General Information...... 365
CanIt-Domain-PRO — AppRiver, LLC CONTENTS 13
H.2 Event Log Format...... 366
I SNMP Agents for CanIt-Domain-PRO 371 I.1 Introduction...... 371 I.2 The SNMP Agent...... 371 I.2.1 Enabling the agent...... 372 I.2.2 Configuring SNMPd...... 372 I.2.3 Agent Data...... 372
J Additional Scripts 375 J.1 reset-password.pl...... 375
K Bayes Database Back-Ends 377 K.1 PostgreSQL Bayes Data Storage...... 377 K.2 Berkeley Database Bayes Storage...... 377 K.3 CDB Database Bayes Storage...... 377 K.4 Cluster Considerations...... 378 K.4.1 Propagating Updates...... 378 K.5 Switching back to PostgreSQL Bayes Storage...... 378
L System Check Tests 379 L.1 Disabling System Checks...... 382 L.2 Anomaly Detection...... 382 L.2.1 Disabling Recipient Verification Anomaly Testing...... 383 L.2.2 More Details about Anomalies...... 384 L.2.3 Suppressing Anomaly Notification Emails...... 384
M The CanIt-Domain-PRO License 385 M.1 THE CANIT DATA LICENSE...... 388
Index 389
CanIt-Domain-PRO — AppRiver, LLC 14 CONTENTS
CanIt-Domain-PRO — AppRiver, LLC List of Figures
2.1 Flow of Mail through CanIt-Domain-PRO...... 28 2.2 RCPT TO: Decision...... 29 2.3 Post-Data Decision...... 30 2.4 Address Streaming...... 34 2.5 Database Agents...... 36
3.1 Administrative Levels...... 39 3.2 Realm Screen...... 40 3.3 Realm Mappings...... 41 3.4 Realm Hierarchy Example...... 44 3.5 Realm Custom Fields...... 45
4.1 Streaming Scenarios...... 49
5.1 License Key Screen...... 53 5.2 Login Screen...... 54 5.3 Welcome Screen...... 55 5.4 Verification Server Operation...... 57 5.5 Verification Servers...... 58 5.6 Domain Routing Screen...... 60 5.7 Domain Routing Detail...... 61 5.8 Cluster Management Page...... 63 5.9 Known Networks...... 65 5.10 Known Network with Associated Domains...... 68 5.11 Rate-Limiting Rules...... 71 5.12 System Check...... 76 5.13 Templates...... 77 5.14 Theme Customizations...... 79
CanIt-Domain-PRO — AppRiver, LLC 15 16 LIST OF FIGURES
5.15 Theme Customization Editor...... 80 5.16 Domain Mappings...... 82 5.17 Address Mappings...... 84 5.18 Domain Overview Page...... 87 5.19 Autotask Product List...... 89 5.20 Autotask Recurring Service Contract...... 90 5.21 Autotask Integration Settings...... 91 5.22 Autotask Test Results...... 94 5.23 Autotask Contract Costs...... 95 5.24 CanIt-Inbound ConnectWise Product...... 96 5.25 CanIt Product List...... 97 5.26 Integrator Login ID Setup...... 98 5.27 CanItBilling Management IT Solution Setup...... 99 5.28 CanItBilling Managed Device Integration Setup...... 100 5.29 Connectwise Agreement...... 101 5.30 Connectwise Agreement Addition...... 102 5.31 ConnectWise Setup - Main Realm...... 103 5.32 ConnectWise Test Results...... 104
6.1 Global Settings...... 105 6.2 Master RBLs...... 110 6.3 Phishing URL Votes...... 113 6.4 Known Phishing URLs...... 114 6.5 Users...... 116 6.6 Add User...... 118 6.7 Edit User...... 119 6.8 Granting Access to Streams...... 120 6.9 Stream Opt-In Approval...... 121 6.10 Groups...... 122 6.11 Group Members...... 123 6.12 Active Streams...... 124 6.13 Known Network with Associated Domains...... 126 6.14 DKIM Key list...... 126 6.15 Adding a DKIM Key Pair...... 127 6.16 DKIM Key Details...... 127
CanIt-Domain-PRO — AppRiver, LLC LIST OF FIGURES 17
6.17 Copying Rules...... 129 6.18 Test Plugins...... 131 6.19 Block Delivery Status Notifications Page...... 134 6.20 Provisioning Information...... 135
7.1 User Lookup List...... 138 7.2 User Lookup Wizard...... 138 7.3 User Lookup: Method Selection...... 138 7.4 IMAP/POP3 User Lookup...... 140 7.5 LDAP User Lookup...... 142 7.6 Azure Active Directory Main Screen...... 146 7.7 Azure Active Directory Application Registration...... 147 7.8 Azure Active Directory Application Settings...... 148 7.9 Azure API Access Settings...... 149 7.10 Azure Read Directory Permission...... 150 7.11 Azure API Key...... 151 7.12 Azure Setup within CanIt-Domain-PRO...... 152 7.13 Program User Lookup...... 153 7.14 Authentication Mappings...... 158 7.15 Cached Logins...... 159
9.1 Permissions Page...... 167 9.2 Permissions Page...... 167 9.3 Stream Permissions Page...... 168 9.4 User Permissions Page...... 170 9.5 Permission Grantability...... 171 9.6 Grantable Permissions Detail...... 172
10.1 Stream Inheritance Terminology...... 174 10.2 Stream Inheritance Table...... 174 10.3 Special Stream Table...... 175 10.4 Simplified Interface...... 176
11.1 Periodic Reports...... 181 11.2 Add Periodic Report...... 182
13.1 Delayed Attachments...... 188
CanIt-Domain-PRO — AppRiver, LLC 18 LIST OF FIGURES
13.2 Attachment-Stripping Rules...... 189
14.1 Redirected Link...... 191 14.2 URL Proxy Rules...... 192
15.1 SMTP Session...... 196 15.2 SMTP Server Test Parameters...... 197 15.3 SMTP Server Test Results...... 198
16.1 CanIt Storage Manager...... 202 16.2 Storage Manager Configuration...... 204
17.1 Log Search Page...... 210 17.2 Saved Log Searches...... 213 17.3 Log Search Results...... 214 17.4 Log Search Details...... 215 17.5 Log Forwarding Page...... 216
A.1 Domain Configuration: Enter Domain Name...... 227 A.2 Domain Configuration: Enter Realm Name...... 228 A.3 Domain Configuration: Configuring Streaming...... 228 A.4 Domain Configuration: Configuring Authentication...... 229 A.5 Domain Configuration: Configuring Routing and Verification...... 230
C.1 Network Configurations...... 332
D.1 CanIt-Domain-PRO Architecture...... 338
L.1 Anomaly Notice...... 382 L.2 Anomaly Details...... 384
CanIt-Domain-PRO — AppRiver, LLC Chapter 1
Introduction
CanIt-Domain-PRO is server-based anti-spam software that stops spam from entering your network. This guide explains how to administer CanIt-Domain-PRO, and is intended for e-mail administrators. For installation instructions, please see the Installation Guide, and for end-user instructions, see the User’s Guide.
1.1 Principles of Operation
CanIt-Domain-PRO uses many sophisticated rules and mechanisms to detect spam. These rules in- clude those in an open-source anti-spam package, and are very effective and broad-spectrum. Once CanIt-Domain-PRO decides that a message is probably spam, it is held for review. A more complete description of how CanIt-Domain-PRO operates is given in Chapter2.
1.2 Handling False-Positives
Although CanIt-Domain-PRO’s rules for identifying spam are very accurate, no purely automated pro- cess can be 100% correct. That is why CanIt-Domain-PRO relies, in the end, on human intervention. In this way, it can guarantee that no legitimate e-mail message will ever be rejected, and you will never lose an important e-mail because of automated scanning. At first glance, it seems that requiring human intervention is a step backwards—spam messages again must be reviewed by a person. In reality, CanIt-Domain-PRO still saves time and money for the following reasons:
• CanIt-Domain-PRO includes many features to lower your workload. (These features are de- scribed later in this manual.) You can scan and categorize e-mail messages using CanIt-Domain- PRO much more quickly than using mail reader software.
• As time passes, you will begin recognize mailing-list traffic and other traffic that tends to be falsely flagged as spam, and tell CanIt-Domain-PRO to always allow that traffic. Over time, this reduces the amount of human intervention required.
CanIt-Domain-PRO — AppRiver, LLC 19 20 CHAPTER 1. INTRODUCTION
• If you are willing to take the risk of inappropriately rejected messages, you can configure CanIt- Domain-PRO to automatically reject very high-scoring messages.
1.2.1 Spam-Control Delegation
CanIt-Domain-PRO operates similarly to CanIt-PRO, except that it allows two levels of administrative delegation. In CanIt-PRO, the system administrator can create separate streams. Stream owners can review quarantined mail within their streams. Only the single system administrator can create streams. In CanIt-Domain-PRO, however, the system administrator creates realms, each of which has its own Realm Administrator. Realm Administrators, in turn, can create streams, each of which has a Stream Owner responsible for settings within the stream. Settings in different streams do not affect other streams.
1.3 Organization of this Manual
This manual is divided as follows: Chapter1, “Introduction”, is this chapter. You should familiarize yourself with the terms in Section 1.4 before proceeding. Chapter2, “Operation”, describes the principles behind CanIt-Domain-PRO’s operation. Chapter3, “Realms”, describes Realms. A Realm is a complete administrative unit in CanIt-Domain- PRO. You must read and understand this chapter before using CanIt-Domain-PRO in production. Chapter4, “Streams”, describes the concepts behind streaming. You must read and understand this chapter before using CanIt-Domain-PRO in production. Chapter5, “CanIt-Domain-PRO Setup”, describes basic setup steps you need to take to configure CanIt-Domain-PRO. Chapter6, “CanIt-Domain-PRO Administration”, describes tasks undertaken by the CanIt-Domain- PRO administrator. Chapter7, “External Authentication”, describes how to integrate CanIt-Domain-PRO with an external authentication mechanism (such as LDAP or POP3.) Chapter8, “Bayesian Filtering”, explains CanIt-Domain-PRO’s Bayesian filtering module. Bayesian filtering uses statistical analysis and training so that CanIt-Domain-PRO “learns” to recognize spam based on user feedback. Chapter9, “Permissions”, describes how to control access to various parts of the CanIt-Domain-PRO Web interface. Chapter 10, “Streams, Inheritance and the Simple GUI”, describes how the CanIt-Domain-PRO ad- ministrator can set up different groups of spam-handling settings and allow end-users to select from one of a limited number of predetermined setups. The simplified interface is very useful if you wish to provide “canned” settings for unsophisticated users. Chapter 12, “Locked Addresses”, describes how CanIt-Domain-PRO permits users to generate ad- dresses that they can give out to strangers, but that those strangers cannot in turn give or sell to
CanIt-Domain-PRO — AppRiver, LLC 1.4. DEFINITIONS 21
third-parties. Chapter 13, “Attachment Handling”, describes CanIt-Domain-PRO options for handling various at- tachments. Chapter 14, “URL Proxying”, describes a CanIt-Domain-PRO feature that can help mitigate phishing attacks that trick users into visiting hostile web sites and entering sensitive information. Chapter 15, “SMTP Server Testing”, describes a CanIt-Domain-PRO feature that lets you run a de- bugging SMTP session against a back-end mail server. Chapter 17, “Searching Logs”, describes CanIt-Domain-PRO’s log-indexing and searching feature (available only on appliance builds.) Chapter 18, “Tips”, contains guidelines for reducing the workload of the spam-control officer and dealing with spam more effectively. Chapter 19, “Security”, contains information about CanIt-Domain-PRO security. AppendixC, “A Testing Topology for CanIt-Domain-PRO”, gives tips on how to test CanIt-Domain- PRO before putting it into production. This appendix also contains useful information on production network topology, so if you are planning on using CanIt-Domain-PRO as a relay-only server, you should read this appendix. AppendixD, “CanIt-Domain-PRO Architecture”, discusses CanIt-Domain-PRO’s filter architecture in detail. It provides tips on tuning CanIt-Domain-PRO and describes the various configuration files used by CanIt-Domain-PRO. AppendixE, “CanIt-Domain-PRO HOWTOs”, gives short “how-to” recipes for performing common CanIt-Domain-PRO administrative tasks, such as restoring a database from the text dump, or moving CanIt-Domain-PRO to another machine. AppendixH, “CanIt-Domain-PRO Logging”, explains how CanIt-Domain-PRO logs statistics, warn- ing, and error messages. AppendixJ, “Additional Scripts”, describes some additional scripts bundled with CanIt-Domain-PRO that you might find useful.
1.4 Definitions
We use many terms related to Internet e-mail in this manual. Here is a definition of some of the terms we use.
Allow list A list of domains, senders or hosts whose e-mail is permitted through without spam- scanning.
API Application Programming Interface. In the context of CanIt-Domain-PRO, the API is a method for interacting with CanIt-Domain-PRO from a program or script.
Backscatter Unwanted DSNs (see “DSN”) caused when e-mail systems respond to faked sender addresses.
Bayesian Analysis is a method whereby an anti-spam system keeps track of how often words appear
CanIt-Domain-PRO — AppRiver, LLC 22 CHAPTER 1. INTRODUCTION
in spam and non-spam. Once enough statistics have been accumulated, the system can calculate the likelihood that a new message is spam.
Blocklist A list of domains, senders or hosts that are blocked from sending e-mail.
CIDR “Classless Inter-Domain Routing”. A method for specifying an entire set of contiguous IP addresses.
CanIt-Domain-PRO is an enhanced version of CanIt-PRO that allows two levels of delegation of responsibility. See the next three definitions for more details.
CanIt-PRO is an enhanced version of CanIt that allows flexible delegation of spam-control respon- sibilities rather than requiring a single spam-control officer.
CanIt is extra software built on top of MIMEDefang that provides sophisticated spam-management functions.
Cron A UNIX program that runs tasks periodically.
DKIM “DomainKeys Identified Mail”. A mechanism for proving that a particular organization’s servers have relayed an email message. DKIM uses cryptographic techniques to assert that a particular domain name is responsible for relaying the message. For more information, see http://www.dkim.org/.
DMARC “Domain-based Message Authentication, Reporting and Conformance”. A mechanism for allowing domain owners to specify a policy that recipients should use in response to potentially- spoofed messages from that domain. For more information, see https://dmarc.org/.
DNS “Domain Name System”. The mechanism used on the Internet to translate host names to IP addresses and more generally, to associate various sorts of information with domain names.
DNSBL “DNS Blocklist”. A DNS-based system for checking in real-time whether or not hosts or domains should be blocked. Sometimes referred to as “Real-time Blocklist” or RBL.
DSN “Delivery Status Notification”. A message generated automatically to notify senders of prob- lems or failure to deliver an e-mail.
Daemon A long-running UNIX program that typically starts at system boot and continues running in the background until the system is shut down. Roughly corresponds to a “service” on Windows.
Envelope Mail messages often have headers specifying the sender (the “From:” header) and recipi- ents (typically the “To:” header.) However, SMTP has a completely separate set of commands for specifying the sender and recipients. The sender and recipients specified in the SMTP com- mands are referred to as the envelope sender and envelope recipients, and do not necessarily match the information in the message headers. CanIt-Domain-PRO uses both the Header From and Envelope Sender address in Sender and Domain rules. It always uses only Envelope Recip- ients in its recipient rules.
Envelope Sender The sender address used in the “MAIL FROM” SMTP command. This is not necessarily the same as the Header From address. Most email readers display the Header From address rather than the Envelope Sender address.
CanIt-Domain-PRO — AppRiver, LLC 1.4. DEFINITIONS 23
Hash An algorithm that computes a short “signature” given a chunk of data. Different inputs are very likely to yield different signatures, so that a signature can be considered as a short-hand identifier for the original data.
Header From The sender address used in the “From:” header of an email message. This is the sender address displayed by most mail readers. See Envelope Sender for information about the SMTP sender address.
Greylisting A technique to block spam from certain spam-sending software. It works by issuing a Temporary Failure Code the first time an e-mail arrives from an unknown sender and IP address. Legitimate SMTP servers will retry, allowing the message to be delivered. Some spam-sending software does not retry, and messages sent by such software will be blocked without any content- scanning if greylisting is enabled.
Joe-Job A technique in which spammers fake the sending address to be that of an innocent victim, who often receives DSNs (see “DSN”) and complaints.
Malware is software designed with a malicious purpose in mind. Examples of malware are viruses, trojans, and keyloggers.
MIMEDefang is a free (GPL’d) e-mail scanning program that integrates with Sendmail’s Milter API. It forms the basis for CanIt.
MIME “Multipurpose Internet Mail Extensions”. A set of rules for encoding different types of at- tachments as plain-text messages for transmission over SMTP.
Milter is a Sendmail interface that allows external programs to listen in on the SMTP dialog, and potentially modify Sendmail’s actions and SMTP responses.
Permanent Failure Code Also called reject, this is a code sent to a relay host telling it that e-mail transmission has failed and will not succeed. (For example, this code is sent if someone tries to send e-mail to a nonexistent user.) The relay host typically e-mails a failure notification to the original sender and discards the message.
Phishing An attack in which someone forges e-mail pretending to be from a security organization, a bank, etc. and convinces naive users to reveal sensitive information like user-names and passwords.
PostgreSQL A free and open-source SQL database heavily used by CanIt-Domain-PRO.
Ransomware is a specific type of malware. It typically makes changes on your computer that are almost impossible to undo (such as encrypting all your files) and then demands payment within a short period of time to undo the damage.
Ratware is software dedicated to sending out large volumes of spam.
RBL “Real-time Blocklist”. A DNS-based system for checking in real-time whether or not hosts or domains should be blocked. Sometimes referred to as “DNS Blocklist” or DNSBL.
CanIt-Domain-PRO — AppRiver, LLC 24 CHAPTER 1. INTRODUCTION
RPTN is the Roaring Penguin Traning Network. (Roaring Penguin was acquired by AppRiver, LLC, so the name is of historical origin.) This is a system whereby multiple CanIt-Domain-PRO installations can share Bayes training data.
RSS stands for “Really Simple Syndication” and is a format for publishing “news feeds” on the Web. CanIt-Domain-PRO can produce an RSS feed showing pending incidents.
Realm Administrator is a user with administrative privileges in a realm. Unlike the System Admin- istrator, a Realm Administrator can only administer his or her own realm.
Realm is a “virtual CanIt-PRO”. Within a realm, realm administrators can create streams for end- users, and streams in one realm are independent of streams in another realm.
Relay Host When a mail server wishes to transmit e-mail to your server using SMTP, it establishes a connection with your mail server. The machine attempting to transmit mail to your server is called a relay host.
REST Representational State Transfer. An architectural style for interacting with an API over HTTP or HTTPS. CanIt-Domain-PRO’s API is REST-based.
Root Privileges A CanIt-Domain-PRO user with root privileges can create other users and configure basic operating parameters. Also, he or she can edit other users’ preferences and stream settings.
SMTP Dialog During the course of e-mail transmission, the two ends of an SMTP connection trans- mit commands and results back and forth. This conversation is called the SMTP dialog.
SMTP “Simple Mail Transfer Protocol”, as described in Internet RFC 2821. This is the protocol used to transmit e-mail over the Internet.
SPF stands for “Sender Policy Framework”. It is a mechanism that allows a domain’s administrator to list which hosts are allowed to originate e-mail claiming to come from that domain. For more details, please see http://www.openspf.org.
SRS stands for “Sender Rewriting Scheme”. It is used in conjunction with SPF to avoid spurious SPF failures when a CanIt-Domain-PRO machine forwards mail to a back-end server that performs SPF checks. For a description of SRS, please see http://en.wikipedia.org/wiki/ Sender_Rewriting_Scheme.
Sender’s Domain This is the domain part (everything after the @ sign) in the sender’s e-mail address.
Sendmail A UNIX-based program for sending and receiving e-mail. Sendmail is designed to route mail from one mail server to another.
Spam Score A numerical score computed by CanIt-Domain-PRO that rates the likelihood that a mes- sage is spam.
Stream is a “virtual CanIt” machine offered by CanIt-PRO. If an incoming e-mail arrives for more than one recipient, and the recipients each wish to have his or her own private spam quarantine, CanIt-PRO re-mails the original message so each recipient has his or her own copy, and can dispatch it as he or she sees fit.
CanIt-Domain-PRO — AppRiver, LLC 1.4. DEFINITIONS 25
Syslog A UNIX program that centralizes the logging of messages from various system daemons.
System Administrator is a user with administrative privileges in the base realm. The System Ad- ministrator is responsible for overall administration of the CanIt-Domain-PRO installation.
Tempfail See “Temporary Failure Code”
Temporary Failure Code Also called tempfail, this is a code sent to a relay host telling it that e-mail transmission has failed temporarily, and it should retry in a little while. Typically, the relay host retains the e-mail message in a spool directory and retries transmission periodically. The host eventually gives up after a certain period (typically, a few days) has elapsed without successful transmission.
Ticker A CanIt-Domain-PRO program that runs periodic maintenance tasks.
Ticker Host In a CanIt-Domain-PRO cluster consisting of more than one machine, exactly one host is designated to run the Ticker tasks. That host is called the Ticker Host.
CanIt-Domain-PRO — AppRiver, LLC 26 CHAPTER 1. INTRODUCTION
CanIt-Domain-PRO — AppRiver, LLC Chapter 2
Operation
2.1 Principles of Operation
CanIt-Domain-PRO watches each incoming SMTP message and operates as follows. Because differ- ent recipients can have different settings, CanIt-Domain-PRO makes the following decisions at RCPT time (once the recipient is known):
• If the SMTP connection is from a blocked host, the RCPT command is rejected. • If the message sender is blocked (or the domain is blocked), the RCPT command is rejected. • Otherwise, the message is collected and scanned.
After CanIt-Domain-PRO has scanned the message, it performs the following operations:
• Messages containing dangerous files (such as viruses) are discarded or rejected, depending on which option you choose. • If the sender, relay host or domain are always-allowed, the message is accepted without being scanned for spam. • Many spam-detection rules are applied to the message. If the message is judged not to be spam, it is accepted and the SMTP transaction succeeds. Otherwise, CanIt-Domain-PRO will hold the message locally.
For messages judged to be spam, CanIt-Domain-PRO takes the following steps:
• A unique ID is calculated by running the message body through a special hash function. The hash calculation is designed to be resistant to some forms of trivial message modification. • The ID is looked up in a database. 1. If the ID is not found in the database, it is entered as a pending message. CanIt-Domain- PRO will either hold a copy of the message locally or send a temporary failure code to the SMTP sender, depending on how CanIt-Domain-PRO has been configured.
CanIt-Domain-PRO — AppRiver, LLC 27 28 CHAPTER 2. OPERATION
2. If the ID is in the database with status pending, CanIt-Domain-PRO may either save a local copy or return a temporary failure code to the SMTP sender, depending on how CanIt-Domain-PRO has been configured. 3. If the ID is in the database with status spam, a permanent rejection code is sent to the SMTP sender. 4. If the ID is in the database with status not-spam, the message is accepted for delivery.
The flow of mail through CanIt-Domain-PRO is summarized in Figure 2.1. Note that this is the conceptual flow; in reality, several optimizations are performed that would only complicate the figure. See also Figures 2.2 on page 29 and 2.3 on page 30 for more accurate details about block and allow rules.
RCPT Command End of DATA
Y Reject Y Discard Block rule? RCPT Virus? Message
N N
Accept RCPT Y Deliver Allow−always? Message
Proceed N to DATA
Looks Y Hold Like Spam? Message
N
Deliver Message
Figure 2.1: Flow of Mail through CanIt-Domain-PRO
2.2 Interaction between Allow Rules and Block Rules
CanIt-Domain-PRO must prioritize allow and block rules. For example, suppose a sender is always allowed, but the host the message comes from is blocked. What should CanIt-Domain-PRO do?
2.2.1 RCPT TO: Actions
At the SMTP RCPT TO: command, CanIt-Domain-PRO examines the envelope sender and SMTP relay address, and makes decisions according to Figure 2.2.
CanIt-Domain-PRO — AppRiver, LLC 2.2. INTERACTION BETWEEN ALLOW RULES AND BLOCK RULES 29
Start
Y Invalid REJECT Recipient?
Relay Y REJECT Blocked?
N N
REJECT Y Sender Blocked?
Relay Y ALLOW N Always−allowed?
N
ALLOW Y Sender Always−allowed?
Relay on Y REJECT N Reject RBL?
N
REJECT Y Domain Blocked?
ALLOW N
ALLOW Y Domain Always−allowed?
N
Figure 2.2: RCPT TO: Decision
Here are the steps illustrated in Figure 2.2. They determine the response to the RCPT TO: command. The first rule that matches returns the result; subsequent rules are not tested.
1. If the recipient is blocked, the command is rejected. Blocked recipients can never receive e- mail.
2. If the recipient has opted out of spam-scanning, the command is accepted.
3. If the sender address is blocked, reject the command with an SMTP failure code.
CanIt-Domain-PRO — AppRiver, LLC 30 CHAPTER 2. OPERATION
4. If the sender address is always allowed, accept the command. (That is, permit the SMTP trans- action to continue. The message may be rejected later for other reasons.)
5. If the domain of the sender is blocked, reject the command.
6. If the domain of the sender is always allowed, accept the command.
7. If the sending relay’s IP address is blocked, reject the command.
8. If the sending relay’s IP address is always allowed, accept the command.
9. If the sending relay is on a real-time blocklist for rejection, then reject the command.
10. Otherwise, accept the command.
2.2.2 Post-DATA Actions
After the SMTP “DATA” command has transmitted the entire message, CanIt-Domain-PRO has enough information to determine a spam score. At this point, it makes decisions according to Fig- ure 2.3.
START
Y Y Y Virus Found? Domain "Hold" Virus Handling Accept Message Always allowed? Hold in Trap RBL Rule?
N N N
Y Y Y High Spam Bad MIME type Domain Hold, Tag Score? Bad Attachment or Extension? Reject Message Blocked? Handling or Reject
N N N
Y Sender Y Domain Accept Message Always allowed? Hold in Trap "Hold"? Accept Message
N N
Y Sender Y Relay Reject Message Blocked? Accept Message Always allowed?
N N
Y Sender Y Relay Hold in Trap "Hold"? Reject Message Blocked?
N N
Y Relay Hold in Trap "Hold"?
N
Figure 2.3: Post-Data Decision
CanIt-Domain-PRO — AppRiver, LLC 2.2. INTERACTION BETWEEN ALLOW RULES AND BLOCK RULES 31
Here are the steps illustrated in Figure 2.3. They determine the response to the DATA command. The first rule which matches returns the result; subsequent rules are not tested. (There is one exception: If a “Hold Sender”, “Hold Domain” or “Hold Relay” rule is hit, but the message scores over the auto-reject threshold, the message is rejected rather than held for review.) When a message is “held in the quarantine”, the message will be held by CanIt-Domain-PRO for review. To the sending SMTP relay, it appears as if the message was delivered successfully. When a message is “rejected”, the sending relay receives an SMTP failure code. If the message being rejected was held within CanIt-Domain-PRO, it is simply discarded. When a message is “accepted”, it is simply delivered as usual.
1. If a virus was found in the message, then the action depends on the virus-handling setting. Here’s what happens for the various settings:
• Hold/Tag – the message is held in the quarantine (or tagged in a tag-only stream.) • Reject – the message is rejected with an SMTP failure code. • Discard – the message is discarded. An SMTP success code is returned. • Accept – processing continues to step (2) below.
2. If a bad MIME part or filename extension was found, then if the bad part has a “Reject” setting, the message is rejected. Otherwise, the message is held in the quarantine (or tagged in a tag-only stream.)
3. If the user has opted-out of spam-scanning, the message is accepted
4. If the sender is always allowed, the message is accepted.
5. If the sender is blocked, the message is rejected. It may seem superfluous to check for a block here, given that the block was checked during the RCPT command. However, by the DATA command, we have the From: header, and CanIt-Domain-PRO applies sender checks to the From: header address also.
6. If the sender has a “Hold/Tag” setting, the message is held in the quarantine (or tagged in a tag-only stream.) However, if it scores over the auto-reject threshold, it will be rejected.
7. If the domain is always allowed, the message is accepted.
8. If the domain is blocked, the message is rejected. Again, at this point, CanIt-Domain-PRO can make use of the From: header address.
9. If the domain has a “Hold/Tag” setting, the message is held in the quarantine or tagged. How- ever, if it scores over the auto-reject threshold, it will be rejected.
10. If the relay is always allowed, the message is accepted.
11. If the relay has a “Hold/Tag” setting, the message is held in the quarantine or tagged. However, if it scores over the auto-reject threshold, it will be rejected.
CanIt-Domain-PRO — AppRiver, LLC 32 CHAPTER 2. OPERATION
12. If the relay is on a “Hold/Tag” real-time DNS blocklist, the message is held in the quarantine or tagged.
13. If CanIt-Domain-PRO is in “Tag Only” mode, the message is tagged (if it looks like spam) and accepted.
14. If the spam score is equal to or above the auto-reject threshold, the message is rejected. Oth- erwise, if the spam score is equal to or above the spam threshold, the message is held in the quarantine.
15. Otherwise, the message is accepted.
2.3 Streaming
Because CanIt-Domain-PRO allows different recipients to have different spam-processing rules, an incoming message for more than one recipient must be streamed. The diagram in Figure 2.1 shows what happens to messages after they have been streamed. If an incoming message arrives for more than one stream, copies are re-mailed to recipients in each stream, and the original message is discarded. Then, each re-mailed message follows the flow in Figure 2.1, with some minor differences that will be explained later. In Figure 2.1, all of the block and allow decisions are unique to a stream. It is perfectly feasible for one stream to always allow a sender, a second stream to block it, and a third stream to do neither. Messages that are streamed and re-mailed are not held by issuing a temporary-failure code, because they would then reside in your own mail spool and waste resources during repeated sending attempts (until they are approved or rejected.) Instead, held messages are stored in the database, and re-mailed if approved or discarded if rejected.
2.4 How Addresses are Streamed
CanIt-Domain-PRO can map e-mail addresses to streams using the following techniques:
Database CanIt-Domain-PRO maintains a table of address-to-stream mappings in the Address Map- ping Table. If you choose the Database technique, then this table is consulted to perform the mapping. You hand-enter the mappings between addresses and streams. In addition, the Database technique allows a “wildcard” lookup if the original lookup does not exist.
AsIs This method simply uses the entire e-mail address as the stream name, after stripping angle- brackets and converting to lower-case. Therefore, [email protected] gets mapped to [email protected],
ChopDomain This method simply chops the domain part off the e-mail address. Therefore, [email protected] gets mapped to xzyyz.
ChopUser This method chops the user part off the e-mail address. Therefore, [email protected] gets mapped to example.com.
CanIt-Domain-PRO — AppRiver, LLC 2.5. HOW STREAMING METHODS ARE CHOSEN 33
Program This method runs the account-info program to determine the stream. Please see Sec- tion 7.2.5 on page 157 for details.
User Lookup You can create so-called “User Lookups” that permit you to use LDAP or arbitrary scripts to map addresses to streams. These are described in Section 7.2.
Note: No matter what stream method you choose, an exact-match database lookup is always done first. This lets you override the mapping for special cases. For example, if you host only a single domain, then the ChopDomain method is probably fine for most addresses. However, if you also host mailing lists, you’d like to stream spam for the lists to the mailing list owners. In that case, you can add special mappings mapping [email protected] to joe-owner, (where joe-owner is the person responsible for list-name.) Because the Program method is somewhat inefficient, CanIt-Domain-PRO caches results in the database table. This improves efficiency while retaining flexibility. By default, cached entries are valid for 24 hours, but you can adjust the timeout.
2.5 How Streaming Methods are Chosen
Each domain can be streamed using its own method. To select a streaming method, CanIt-Domain- PRO first looks up the domain in the Domain Mapping Table. This table holds a list of streaming methods for each domain. If the lookup fails, CanIt-Domain-PRO looks up the wildcard entry “*” in the Domain Mapping Table and uses that method to stream the address. Figure 2.4 illustrates how addresses are streamed.
CanIt-Domain-PRO — AppRiver, LLC 34 CHAPTER 2. OPERATION
Incoming Mail for stream = lookup [email protected] "[email protected]" (followed by "user@*" if not found) in Address Mapping Table
method = lookup "example.com" in Domain Mapping Table Y stream found?
N
Y method found?
method = ChopDomain Y N ChopUser stream = adjust address or AsIs?
N method = lookup "*" in Domain Mapping Table
method = Y Run account−info script Cache stream in Program? to determine Address Mapping local user Table
Y N method found?
N method = Y Look up stream LDAP in LDAP directory. Return stream method = "Database" N
stream = lookup "*@example.com" in Address Mapping Table
Y stream found?
N
stream = lookup "*" in Address Mapping Table
Y stream found?
N
stream = "default"
Figure 2.4: Address Streaming
CanIt-Domain-PRO — AppRiver, LLC 2.6. STATUS OF MESSAGES 35
Figure 2.4 looks complicated, but the streaming process is very flexible, and actually quite simple. Here is a description of the figure, with some more details that would crowd the figure too much.
1. For an incoming message to [email protected], CanIt-Domain-PRO first looks up exam- ple.com in the Domain Mapping Table. If that lookup succeeds, CanIt-Domain-PRO will have a method (ChopDomain, ChopUser, Program, Database or a user-lookup name), and CanIt- Domain-PRO proceeds to Step4.
2. If the lookup fails, the leading component of the domain name is dropped (ie: “subdo- main.example.com” becomes “example.com”) and we retry Step1 with the shorter name.
3. If lookups on all domain components fail, CanIt-Domain-PRO looks up * in the Domain Map- ping Table. This allows you to set a default streaming method for all domains. If that lookup fails, the method defaults to Database.
4. Regardless of the method chosen, CanIt-Domain-PRO looks up [email protected] in the Ad- dress Mapping Table. If an exact match is found (and it is not expired if it is a cached entry), the result of that lookup is used as the stream. If the exact match is not found, but a wildcard user@* is found in the Address Mapping Table, the result of that lookup is used as the stream.
5. Otherwise, CanIt-Domain-PRO determines the stream as follows:
• If the method is ChopDomain, the @example.com part is deleted, and the stream becomes user. • If the method is ChopUser, the user@ part is deleted, and the stream becomes exam- ple.com. • If the method is AsIs, the entire e-mail address [email protected] is used as the stream name. • If the method is Program, CanIt-Domain-PRO runs the account-info program as described in Section 7.2.5. • If the method refers to a user-lookup, then the user-lookup is invoked to determine the stream. See Section 7.2 for details.
If the stream determination succeeded (AsIs, ChopDomain and ChopUser always succeed; Program fails if the program produces no output), then the stream is returned. Additionally, the stream may be cached in the Address Mapping Table.
6. If the previous step failed to determine a mapping method, or the method was set to Database, CanIt-Domain-PRO looks up *@example.com in the address mapping table. This allows you to map all addresses in a particular domain to a stream. If that fails, as a last resort, CanIt-Domain- PRO looks up * in the address mapping table. If that final lookup fails, then a special stream named default is used.
2.6 Status of Messages
Every message in the database has one of three statuses. The status names and their meanings are:
CanIt-Domain-PRO — AppRiver, LLC 36 CHAPTER 2. OPERATION
pending Messages enter pending state when they arrive, and remain there until they are marked as spam or nonspam. These messages are displayed in the Web-based “Pending Messages” list. spam The spam-control officer can mark a message as spam. If a message marked as spam is re- ceived, a rejection notice is sent to the sending mail server, and the message is not delivered. not-spam The spam-control officer can mark a message as not-spam. If a message marked as not- spam is received, it is delivered as usual.
2.6.1 Secondary MX Relays
Many organizations have secondary MX hosts that queue mail if the primary host is down. They then relay the queued mail when the primary MX host comes back up. Ideally, CanIt-Domain-PRO should run on all of your MX hosts. However, if it can only run on your primary MX host, then all other MX hosts should relay to the CanIt-Domain-PRO machine. You should then tell CanIt-Domain-PRO the IP addresses of the secondary MX hosts via the “Known Networks” facility so that CanIt-Domain-PRO can use the Never Tempfail handling for messages from those hosts. (There is no point in keeping mail queued and retransmitted on your secondary MX hosts; it’s better to accept and hold the message on the CanIt-Domain-PRO machine.)
2.7 The Database
The incident database is key to the correct operation of CanIt-Domain-PRO. Three different agents operate on the database as shown in Figure 2.5:
CanIt Filter
Web−Based GUI Periodic Jobs
Incidents Database
Figure 2.5: Database Agents
The agents operating on the database are:
• The CanIt-Domain-PRO Filter – This is the portion of CanIt-Domain-PRO that integrates with Sendmail and disposes of spam messages.
CanIt-Domain-PRO — AppRiver, LLC 2.8. REMAILING MESSAGES 37
• The Web-Based GUI – This is used by users or administrators to mark messages as spam or legitimate. The Web-Based GUI also lets you monitor the levels of spam and take action against specific senders, domains or relay hosts.
• Periodic Jobs – These housekeeping jobs perform operations like moving expired pending mes- sages into spam status and purging very old messages from the database. Periodic jobs may be started from one of two places:
1. The /usr/share/canit/scripts/canit.cron script, which should be run once a night. 2. As part of the operation of the CanIt-Domain-PRO daemon (canitd). Canitd is a daemon that starts on bootup and runs continuously, performing background maintenance tasks.
2.8 Remailing Messages
On occasion, CanIt-Domain-PRO will be forced to remail a message after discarding the original. The following scenarios cause remailing:
1. If a message comes in for recipients in more than one stream, CanIt-Domain-PRO generates one new copy for each stream and mails out the copies. The original message is then discarded. You may see a message in the log file indicating that the message has been discarded; don’t panic. The copies are safely queued.
2. If a Pending message is held in the database and subsequently approved for release, CanIt- Domain-PRO fetches the message body from the database and remails it. This always takes place on the designated ticker host, no matter which host processed the original message.
In all cases when CanIt-Domain-PRO remails a message, the message goes into Sendmail’s submission queue (most likely in the queue directory /var/spool/clientmqueue or /var/spool/mqueue-client. The message is only processed on the next run of the submis- sion queue. For this reason, you should keep the submission queue interval short (on the order of a minute or two.) On CanIt-Domain-PRO appliances, the submission interval is automatically config- ured for you. On other platforms, consult your system’s documentation for details on how to shorten Sendmail’s submission queue interval.
CanIt-Domain-PRO — AppRiver, LLC 38 CHAPTER 2. OPERATION
CanIt-Domain-PRO — AppRiver, LLC Chapter 3
Realms
3.1 Introduction to Realms
CanIt-Domain-PRO has three levels of administrative control:
1. The System Administrator administers all aspects of CanIt-Domain-PRO and is responsible for setting up and provisioning the system.
2.A Realm Administrator administers settings and rules for a given realm. A realm encompasses one or more Internet domains. The realm administrator is responsible for provisioning streams within his or her realm. A realm administrator is said to have root privileges within a realm.
3.A Stream Owner administers settings and rules for his or her own stream. A stream owner is typically an end-user or a person responsible for administering a small group of e-mail ad- dresses.
The administrative levels are illustrated in Figure 3.1 below:
System Administrator
Realm 1 Realm 2 Realm N Realm Administrator Realm Administrator Realm Administrator
Stream 1 Stream N Stream Owner Stream Owner
Stream 1 Stream N Stream Owner Stream Owner
Figure 3.1: Administrative Levels
CanIt-Domain-PRO — AppRiver, LLC 39 40 CHAPTER 3. REALMS
3.2 Realm Names
A realm name can consists only of letters, numbers, dashes and underscores. That is, only the follow- ing characters can appear in a realm name: ABCDEFGHIJKLMNOPQRSTUVWXYZ abcdefghijklmnopqrstuvwxyz 01234567890- Realm names are case-sensitive; a realm named REALM-ONE is different from realm-one.
3.2.1 The base Realm
The realm named base is special. This realm always exists and cannot be deleted. Any user with root privileges in the base realm is considered an overall CanIt-Domain-PRO system administrator, and can access any realm and setting. In other words, a realm administrator of the base realm is an overall CanIt-Domain-PRO administra- tor.
3.3 Creating Realms
Note: This section describes features that only the CanIt-Domain-PRO System Administrator can use. Click on Setup and then Realms. The Realm Screen appears:
Figure 3.2: Realm Screen
To create a realm:
1. Enter the realm name in the Realm box.
2. Enter a short description in the Description box.
CanIt-Domain-PRO — AppRiver, LLC 3.4. REALM MAPPINGS 41
3. If you wish to enter an expiry date, do so in the Expiry box. See Section 3.6 for details about realm expiry.
4. Normally, all realms you create have the base realm as a parent realm. If you wish to set a realm’s parent to something else, select a realm name from the Parent pull-down menu. See Section 3.7 for details about realm hierarchy.
5. Click Submit Changes.
To delete a realm:
1. Enable the Delete? checkbox for the realm you wish to delete.
2. Click Submit Changes.
Note that it is not possible to delete the base realm.
3.4 Realm Mappings
Note: Only the CanIt-Domain-PRO System Administrator can create new realm mappings. Realm adminis- trators can delete realm mappings (irrevocably) or remap a domain from one realm to another. To associate a domain with a realm, CanIt-Domain-PRO uses a Realm Mapping Table. To access this table, click on Setup and then Realm Mappings. The Realm Mappings screen appears:
Figure 3.3: Realm Mappings
In this example, the domains roaringpenguin.com and roaringpenguin.ca are both mapped to the roaringpenguin realm, while artandframingsolutions.com is mapped to afs. If CanIt-Domain-PRO accepts mail for other domains, then they will be mapped to the base
CanIt-Domain-PRO — AppRiver, LLC 42 CHAPTER 3. REALMS
realm. Any domain without an explicit realm mapping will be mapped to base. (The rules for realm mapping are summarized in Section 3.5.) To add a realm mapping:
1. Enter the domain name in the Domain box.
2. Select the realm name in the Realm box. Note that you must create realms before you can add mappings to them.
3. Click Submit Changes.
To delete a realm mapping:
• Enable the checkbox next to the mapping you wish to delete.
• Click Submit Changes.
3.5 Determining the Realm
CanIt-Domain-PRO determines the realm for e-mail addresses and user names as follows:
3.5.1 Mapping a Domain to a Realm
Given a domain such as sub.example.com, CanIt-Domain-PRO searches the Realm Mapping Table in order for the following mappings:
1. sub.example.com
2. example.com
3. com
4. *
and uses the first one that it finds. If no mapping is found, the domain is mapped to the base realm.
3.5.2 Mapping an Address to a Realm
Given an e-mail address of the form [email protected], CanIt-Domain-PRO looks up the domain using the algorithm specified above in Section 3.5.1 using domain.com as the lookup domain.
Note: The addresses postmaster, postmaster@localhost and postmaster@machine name are always mapped to the base realm, no matter what. (Here, machine name is the name of the host processing the email.)
CanIt-Domain-PRO — AppRiver, LLC 3.6. REALM EXPIRY 43
3.5.3 Mapping a Login Name to a Realm
1. If a user’s login name is of the form [email protected], then CanIt-Domain-PRO uses the procedure described in Section 3.5.2 to determine the realm.
2. If a user logs in with a name of the form realm:user, then CanIt-Domain-PRO uses realm as the realm name.
3. Otherwise, CanIt-Domain-PRO uses the default realm as configured in the site/config.php configuration file. If no default realm is set in that file, then CanIt-Domain-PRO uses base as the realm name.
3.6 Realm Expiry
When you create a realm, you can set an expiry date. Whenever the realm administrator logs in to CanIt-Domain-PRO, he or she will receive a warning starting 30 days prior to the expiry date. If you are hosting CanIt-Domain-PRO realms on behalf of third-parties, this is a good way to remind them to renew their subscription. The expiry date normally has no other effect (in particular, CanIt-Domain- PRO will continue filtering mail as usual after the expiry date) and is intended only as a renewal reminder. If you do not set an expiry date, then the realm never expires.
3.6.1 Suspending Service to a Realm
While the expiry date field normally has no effect, if you set the expiry to the “magic” date 1990-01-01, then all service to the realm is suspended. What this means is:
• No users in that realm will be able to log in.
• All mail to anyone in the realm will be permanently rejected with a “Service suspended” error message.
Suspending service to a realm is a drastic step since it causes all mail to bounce. Please use it only as a last resort.
3.7 Realm Hierarchy
Realms normally have the base realm as their parent. However, if you are reselling CanIt-Domain- PRO services to others who wish to have their own set of realms for their customers, you can create a realm hierarchy. A realm administrator has access to his or her own realm and all realms under it. Consider Figure 3.4:
CanIt-Domain-PRO — AppRiver, LLC 44 CHAPTER 3. REALMS
base
cust−1 cust−2
subcust−2−1 subcust−2−2
subcust−2−1−1
Figure 3.4: Realm Hierarchy Example
In the example in Figure 3.4, the parent of cust-1 and cust-2 is base. The parent of subcust-2-1 and subcust-2-2 is cust-2, and the parent of subcust-2-1-1 is subcust-2-1.
• The administrative user in the base realm can access all realms.
• The administrator in cust-1 can only access the cust-1 realm.
• The administrator in cust-2 can access subcust-2-1, subcust-2-2 and subcust-2-1-1.
• The administrator in subcust-2-1 can access subcust-2-1 and subcust-2-1-1.
• The administrator in subcust-2-2 can only access subcust-2-2.
• The administrator in subcust-2-1-1 can only access subcust-2-1-1.
In the Realms screen (Figure 3.2), click on Tree View to see a hierarchical view of the realms. You can restrict the view to a subtree of the entire hierarchy by selecting the root of the tree from the Tree root pull-down menu.
3.8 Realm Custom Fields
CanIt-Domain-PRO allows you to create up to four custom fields so you can associate various pieces of information with a realm. For example, you may wish to include a customer number with each realm. To configure custom fields, click on Setup and then Realms. In the realm display, click on Custom Fields. The Custom Fields screen appears:
CanIt-Domain-PRO — AppRiver, LLC 3.8. REALM CUSTOM FIELDS 45
Figure 3.5: Realm Custom Fields
To create custom fields:
1. Enter the name of the field in the Name box.
2. If you wish to have the field displayed specially, enter a format string in the Format box. This string must contain exactly one %s sequence; this will be replaced by the value of the custom field. In the example in Figure 3.5, Custom Field 2 (AccountID) will be displayed as a hyperlink, presumably to an accounting system.
3. Click Submit Changes to make the changes take effect.
Any custom fields you create are displayed as additional columns in the Realms screen (for the CanIt- Domain-PRO administrator only!). To remove a custom field, simply make the Name column blank.
CanIt-Domain-PRO — AppRiver, LLC 46 CHAPTER 3. REALMS
CanIt-Domain-PRO — AppRiver, LLC Chapter 4
Streams
4.1 Introduction to Streams
The stream is a central concept in CanIt-Domain-PRO. Understanding streams is essential to un- derstanding CanIt-Domain-PRO. Please be sure to read this chapter before configuring a production CanIt-Domain-PRO server.
4.2 Realms
A realm is a collection of Internet domains, all of whose anti-spam settings and quarantines are provi- sioned by a Realm Administrator. Within a realm, there may be many streams. Two streams with the same name can coexist in different realms; CanIt-Domain-PRO will consider them to be two different streams.
4.3 The Definition of a Stream
A stream is a collection of rules and policies. Each stream in CanIt-Domain-PRO can have its own rules, settings, thresholds and policies. Associated with each stream is a quarantine. A quarantine consists of messages that have been held based on the streams settings. For example, a message can be held because of its spam score, or because it contains a suspicious MIME type.
4.4 Users and E-Mail Addresses
Under many circumstances, a single e-mail address corresponds to a single user. For example, the e-mail address [email protected] corresponds to the single user dfs. However, most mail setups are more complicated than this. The first complication comes from aliases. For example, the user dfs may have, in addition to his normal e-mail address, aliases
CanIt-Domain-PRO — AppRiver, LLC 47 48 CHAPTER 4. STREAMS
like [email protected] and [email protected]. We would most likely want the same settings and policies to apply to all three aliases. Another complication comes from list addresses. For example, the e-mail address [email protected] does not correspond to any particular user. Instead, it is a list alias that expands to several users. It might make sense to have a separate set of policies for sales than for real users, or it might make sense to assign the policies used by one of the recipients on the sales list. As we see above, the mapping between users and e-mail addresses is not simple. A single e-mail address may result in delivery to several users (the sales example), or a single user may have several e-mail addresses that all deliver to the same place (the aliases example.) Streams were created to give you the flexibility of assigning policies. They act as an intermediate container between e-mail addresses and actual users, and let you assign policies any way you choose. As an example, consider Figure 4.1:
CanIt-Domain-PRO — AppRiver, LLC 4.4. USERS AND E-MAIL ADDRESSES 49
E−Mail AddressStream User−ID
[email protected] [email protected] dfs dfs [email protected]
[email protected] paul paul
(a)
E−Mail AddressStream User−ID
[email protected] [email protected] dfs dfs [email protected]
[email protected] sales
[email protected] paul paul
(b)
Figure 4.1: Streaming Scenarios
Note that streaming affects only how CanIt-Domain-PRO directs mail for rule and quarantine pur- poses. Streaming does not alter the ultimate delivery address; normally, CanIt-Domain-PRO delivers
CanIt-Domain-PRO — AppRiver, LLC 50 CHAPTER 4. STREAMS
mail to the back-end server without altering recipient addresses at all. We assume that there are two users, dfs and paul. We assume that dfs has the three aliases shown, and that the sales address actually gets delivered to both dfs and paul. In Figure 4.1(a), all mail for dfs’s aliases go into the dfs stream. Mail for paul goes into the paul stream. Furthermore, mail for sales also goes into paul. Although mail for sales is delivered to two users, all of the settings and policies are controlled by the paul stream, and paul is responsible for clearing the quarantine. In Figure 4.1(b), sales has its own stream. It can thus have different settings and rules from either paul or dfs. Furthermore, both paul and dfs are given access to the stream, so either of those users can adjust the settings and check the quarantine for sales.
4.5 Mapping
When e-mail comes in, each recipient address is mapped to a stream. We call this process address mapping. Once the stream is determined, CanIt-Domain-PRO knows which settings and rules to apply for that recipient. The process by which CanIt-Domain-PRO maps addresses to streams is illustrated in Figure 2.4 on page 34. An e-mail address is mapped to a stream in a three-step process:
1. The domain part of the address (everything after the “@” sign) is looked up in the Realm Map- ping Table. This lookup determines the realm to which the address belongs.
2. The domain part of the address is looked up in the Domain Mapping Table. This lookup results in a method by which to map the address to a stream. Note that CanIt-Domain- PRO looks up the mapping method using a combination of the realm (determined in step 1) and the domain. The combination of realm and stream determined in this step is written realm name:stream name
3. Once the method has been determined, then the address is mapped to a stream using the appro- priate method. Details are in Section 5.14 on page 81.
Note: If there is an exact match for an email address in the Address Mapping Table (under Setup : Address-to-Stream Mappings) then it is always used, overriding any mapping method. Fur- thermore, if there is no exact match, but there is an entry for user@*, then that entry too is used, overriding any mapping method.
4.6 The Home Stream
When a user logs in to the Web interface, CanIt-Domain-PRO must associate a stream with the user name. By default, CanIt-Domain-PRO chooses a stream with the same name as the user’s login—this is called the home stream. For example, the user dfs would automatically be sent to the stream dfs upon login. However, it is possible to give users access to additional streams, and to change the default
CanIt-Domain-PRO — AppRiver, LLC 4.7. THE “DEFAULT” STREAM 51
login stream. Also, it is possible to change the user’s home stream with the account-info script (Section 7.2.5).
Note: Stream names are case-sensitive. Thus, a stream called dfs is completely separate from a stream called DFS.
4.7 The “default” Stream
CanIt-Domain-PRO treats the stream named default specially in several ways:
• When the database initialization script runs, it sets the login stream for the CanIt-Domain-PRO administrator to default.
• If a stream mapping cannot be found for an address, the address is mapped to default.
• Any blocks, allow rules and other rules defined in the default stream are inherited by all other streams. (However, stream owners can turn this inheritance off if they wish.) Note that in CanIt-Domain-PRO, rules for a stream example-stream in realm example-realm are searched up through the realm hierarchy.
1. Search for the rule in example-realm:example-stream. 2. If not found, search in example-realm:default. 3. If not found, search in example-realm’s parent realm in the default stream. Con- tinue looking up the realm hierarchy until base:default is reached.
CanIt-Domain-PRO — AppRiver, LLC 52 CHAPTER 4. STREAMS
CanIt-Domain-PRO — AppRiver, LLC Chapter 5
CanIt-Domain-PRO Setup
5.1 Accessing The Web Interface
Using your Web browser, open the URL where you installed the CanIt-Domain-PRO web pages. For example, if your server is mailserver.mydomain.com and you installed the GUI in the directory canit under your Apache document root, the URL to open would be: http://mailserver.mydomain.com/canit/ (By default, our binary packages and our Debian-based appliances put the web pages at http://machine.yourdomain.net/canit/)
5.1.1 License Key Screen
The very first time you log in, you will see the License Key Screen (Figure 5.1):
Figure 5.1: License Key Screen
Enter or cut-and-paste your license key into the entry box and click Submit Key. The license key includes all the text starting from License and continuing to the end of the string of letters and numbers after Check=.
CanIt-Domain-PRO — AppRiver, LLC 53 54 CHAPTER 5. CANIT-DOMAIN-PRO SETUP
5.1.2 Login Screen
Once the license key has been entered, navigating to the CanIt-Domain-PRO URL reveals the Login Screen (Figure 5.2):
Figure 5.2: Login Screen
Log in using the name and password you selected when you initialized the CanIt-Domain-PRO database. (See Section J.1 on page 375 if you’ve forgotten the password.) In the Installation Guide example, we used “admin” and “secret”. If you have a CanIt-Domain-PRO appliance, the defaults are “admin” and “canit”. (Naturally, you should change the password before connecting your CanIt-Domain-PRO appliance to the Internet!) Normally, CanIt-Domain-PRO will set a session cookie in your browser. This means that if you close your browser, your session will automatically end. If you want CanIt-Domain-PRO to remember your session even if you close the browser, enable the “Remember Me” checkbox. This puts a cookie that lasts longer (by default, 7 days) on your computer. Do not use the “Remember Me” option on a public computer; you should only use it on a workstation to which you alone have access. Once logged in, you should see the CanIt-Domain-PRO welcome screen:
CanIt-Domain-PRO — AppRiver, LLC 5.2. THE SETUP MENU 55
Figure 5.3: Welcome Screen
5.2 The Setup Menu
The Setup main menu entry contains sub-entries for various parts of basic CanIt-Domain-PRO setup. Under the Setup menu, you will find:
• Wizards – a collection of tools for easily configuring certain common scenarios.
• License Key – a page to enter your CanIt-Domain-PRO license key.
• Verification Servers – a table allowing you to check recipients against internal servers before CanIt-Domain-PRO will accept them.
• Known Networks – a table allowing you to change aspects of CanIt-Domain-PRO behavior for mail originating from certain known networks.
• Features – a page allowing you to turn off certain CanIt-Domain-PRO functionality to improve performance.
• System Check – a page that performs a few simple “sanity checks” on your CanIt-Domain-PRO system.
• Templates – a page for configuring templates that control how CanIt-Domain-PRO appends Bayesian voting information to e-mail and the format of Pending Message Notifications.
• Theme Customization – a page for customizing the CanIt-Domain-PRO “look”. Can be used to brand CanIt-Domain-PRO.
• Domain Routing – a page for configuring e-mail routing. Please note that this link is available only on Debian-based appliances or on RPM installations with the appliance RPMs installed.
• HTTPS – a page for configuring HTTPS. Please note that this link is available only on Debian- based appliances. (It is not available on RPM or source installations.)
CanIt-Domain-PRO — AppRiver, LLC 56 CHAPTER 5. CANIT-DOMAIN-PRO SETUP
• Cluster Management – a page for viewing and managing cluster members.
• Domain Mappings and Address Mappings – two tables that tell CanIt-Domain-PRO how to convert an e-mail address to a stream.
• Authentication Mappings and User Lookups – pages for integrating CanIt-Domain-PRO with external directories or authentication mechanisms. These are fully described in Chapter7.
5.3 Wizards
Note: This section describes features that only the CanIt-Domain-PRO System Administrator can use. The Wizards menu item allows you to ease CanIt-Domain-PRO setup by using a wizard to speed through choosing some basic settings. The available wizards are shown on the Setup page. The wizards are self-documenting and guide you through the steps required to configure CanIt-Domain- PRO. However, the following wizards are important enough to warrant mention:
5.3.1 Basic Setup Wizard
The Basic Setup Wizard helps you set some basic settings essential to the operation of CanIt-Domain- PRO. On a new CanIt-Domain-PRO installation, you should follow the steps in this wizard to set some basic settings to sensible values. It is important not to operate CanIt-Domain-PRO until you have worked through the Basic Setup Wizard.
5.3.2 RPTN Setup Wizard
The RPTN Setup Wizard configures RPTN, the Roaring Penguin Training Network. (RPTN is a mechanism for sharing Bayes data to increase scanning accuracy. See Section 8.5 on page 162 for details.)
5.3.3 Dictionary Attack Detection Wizard
Note: Dictionary Attack Detection works only on Linux. A Dictionary Attack is an attack whereby an attacker tries to send mail to hundreds or thousands of different e-mail addresses within a domain in the hopes of discovering some valid addresses. CanIt- Domain-PRO (on Linux only) can react to dictionary attacks by blocking them using kernel firewall rules. To enable dictionary-attack detection:
1. Click on Setup : Wizards and then Dictionary Attack Detection Wizard.
2. Select Yes when asked “Would you like to enable the dictionary-attack detector?” Click Next.
3. Adjust the parameters as follows:
CanIt-Domain-PRO — AppRiver, LLC 5.4. VERIFICATION SERVERS 57
• Time span over which to track bad recipients specifies for how long CanIt-Domain- PRO will keep history. For example, if you specify 900 seconds, then CanIt-Domain-PRO tracks bad recipients over the last 15 minutes. • Number of bad recipients to trigger firewalling specifies how many bad RCPT com- mands a host must issue (within the tracking time) to be firewalled off. Continuing the example, if you specify 5 for this parameter, then any host that issues 5 invalid RCPT commands within 900 seconds will be firewalled off. • Length of time in seconds to remain firewalled specifies how long a host remains fire- walled once CanIt-Domain-PRO decides it is an attacker. The default is 3600 seconds (one hour.)
4. Click Next
5. Review your settings and click Finish to make them take effect.
You may wish to exclude certain hosts from ever being banned because of bad RCPT commands. You can exclude such hosts by adding them to the Known Networks list (Section 5.7) with the Omit from Dictionary Attack Detection flag set.
Note: When a host is firewalled off, the Sendmail process that triggered the firewall rule will not receive any traffic from the host. By default, Sendmail will wait one hour between commands. This is far too long if you use the dictionary-attack detector; we recommend shortening Sendmail’s Timeout.command parameter to 5 minutes or shorter. On CanIt-Domain-PRO appliances, this configuration change has been done for you. On other platforms, include the line: define(‘confTO COMMAND’, ‘5m’)dnl in your sendmail.mc file and rebuild sendmail.cf.
5.4 Verification Servers
If CanIt-Domain-PRO acts as a filtering server that always forwards mail on to other machines, you can have it check recipient addresses against other machines. The internal machine that verifies recipient addresses is called a Verification Server. The mechanism is illustrated in Figure 5.4:
1 2 RCPT TO:
Figure 5.4: Verification Server Operation
The sequence of events in Figure 5.4 is as follows:
1. An external SMTP server sends the command: RCPT TO:
CanIt-Domain-PRO — AppRiver, LLC 58 CHAPTER 5. CANIT-DOMAIN-PRO SETUP
2. Before CanIt-Domain-PRO accepts the RCPT command, it starts an SMTP session with the Verification Server (sending a HELO and MAIL command first) and sends the same RCPT command to the Verification Server.
3. The Verification Server responds to the CanIt-Domain-PRO scanner with a reply code.
4. The CanIt-Domain-PRO scanner responds to the external server with the same response it re- ceived from the Verification Server.
Note: This feature only works if the internal machines fail RCPT commands for unknown users. That is, the internal machine must be configured to reject unknown recipients during the SMTP transaction. Some SMTP servers accept any recipient address and then later on generate a failure notification. Servers that delay the rejection of invalid addresses in this manner will not work as Verification Servers. Versions of Microsoft Exchange prior to Exchange 2003 will not work as verification servers. Recent Exchange versions can be configured to reject unknown recipients during the SMTP transaction. See the instructions linked from https://www.roaringpenguin.com/ recipient-verification for your version of Exchange. In all cases, you should disable all other Exchange anti-spam features including tarpitting. (Tarpitting is a completely useless technology for a server behind a spam filter and serves only to slow down CanIt-Domain-PRO.) Make sure that the only anti-spam feature enabled on the Exchange server is recipient filtering. CanIt-Domain-PRO allows you to enter a list of domains and the machines that will verify mail for the domains. (Note that this does not change your Sendmail configuration; you need to ensure that Sendmail’s mailertable routes mail appropriately.) To edit the verification server list, click on Setup and then Verification Servers. The following page appears:
Figure 5.5: Verification Servers
In this example, CanIt-Domain-PRO performs the following checks:
• Any recipient whose domain is blacky.roaringpenguin.com is verified against the ma- chine blacky.roaringpenguin.com
• Any recipient whose domain is canit.ca is verified against the machine mail.canit.ca
CanIt-Domain-PRO — AppRiver, LLC 5.4. VERIFICATION SERVERS 59
• Any recipient whose domain is roaringpenguin.com is verified against the machine mail.roaringpenguin.com
To add a domain/server pair to the table:
• Enter the domain name in the Domain box and the server name or IP address in the Server box. Note that you can enter multiple verification servers in the Server box by separating the names or addresses with commas. If you enter multiple servers, CanIt-Domain-PRO tries them in order until it receive a definite positive or negative response.
• Sometimes, your verification server may be down or unreachable. There are three approaches to deal with this situation:
– If you would like CanIt-Domain-PRO to tempfail the mail, set Action if Unavailable to “Tempfail”. – If you would like CanIt-Domain-PRO to queue mail to addresses that have been proven valid in the last 60 days, set Action if Unavailable to “Queue Seen Addresses”. This is the recommended setting and is the default. – If yo would like CanIt-Domain-PRO to queue all mail (even if the recipients have not been proven valid), set Action if Unavailable to “Queue All Addresses”. Note: This setting runs the risk of causing backscatter and is not recommended.
• Click Submit Changes
To delete a domain/server pair from the table, enable the appropriate Delete checkbox and click Sub- mit Changes. If you enter a string in the “Filter:” box, then CanIt-Domain-PRO limits the display to entries whose Domain or Server columns contain that string. If your verification server listens on a non-standard port (that is, a port other than port 25), you may specify the port number by following the server name with a slash and the number. For ex- ample, if you have a server called mail.example.com that listens on port 2525, you can use mail.example.com/2525 in the Server column.
Note: If you use a verification server, ensure that the server does not throttle or rate-limit the CanIt-Domain- PRO server in any way. Because CanIt-Domain-PRO runs an SMTP connection for each RCPT com- mand, some naive SMTP server software may think it’s under attack and rate-limit the CanIt-Domain- PRO server, with disastrous results.
5.4.1 Wildcard Verification Server
You may optionally choose to add a Verification Server entry for the wildcard domain of ’*’. This will cause mail for any domain that does not have a specific entry to be checked against that server.
Note: If you are relaying outbound mail via your CanIt-Domain-PRO server, you should NOT use a wild- card verification entry, as it will likely result in the rejection of all outbound mail. You can avoid this
CanIt-Domain-PRO — AppRiver, LLC 60 CHAPTER 5. CANIT-DOMAIN-PRO SETUP
problem by forcing outbound mail through a different realm than inbound mail; in this way, the in- bound realm’s verification server settings are not used for outbound mail. The outbound realm may be a subrealm of the inbound realm if you wish to provide administrative access to the inbound realm’s administrator. Finally, note that Verification Server lookups are made only on CanIt-Domain-PRO cluster members that are marked “Inbound” in the Cluster Members Table. (Normally, all CanIt-Domain-PRO nodes are marked “Inbound”.)
5.4.2 SRS and Verification Servers
CanIt-Domain-PRO will rewrite the envelope sender using SRS before checking against a Verification Server if all of the following conditions are met:
1. SRS has been configured.
2. The sending address SPF lookup resulted in “pass”.
3. The quarantine setting “Enable SRS” is set to true in the default stream of the recipient’s realm. The reason CanIt-Domain-PRO looks in the default stream is that address-to-stream mapping is normally only done after a recipient address has been verified.
5.5 Mail Routing
Note: This section is applicable only to Hosted CanIt, to Debian-based CanIt-Domain-PRO appliances or to Red Hat installations with the appliance RPMs installed. On other CanIt-Domain-PRO installations, you need to configure routing manually by editing Sendmail’s access and mailertable files. Please note the following important requirement: All of the features in Sections 5.5 through 5.6 rely on SSH to operate. Your system must be running an SSH server listening on port 22 and it must allow public-key authentication. If you are running a cluster, all cluster members must be running an SSH server on port 22 and permit connections from all other cluster members. If your SSH server listens on a different port, the features will not work. To configure mail routing, click on Setup and then Domain Routing. The Domain Routing page comes up:
Figure 5.6: Domain Routing Screen
CanIt-Domain-PRO — AppRiver, LLC 5.5. MAIL ROUTING 61
Note that the Domain Routing page shows the routing for all domains in the current realm and in all of its subrealms. To add a domain for routing:
1. Enter the domain name in the “Domain” box.
2. Click Add Domain
The Domain Routing Detail screen will come up:
Figure 5.7: Domain Routing Detail
1. Enter the servers to which mail should be routed for the given domain. You can enter more than one server; if you need more than one, enter them one per line. The servers are tried in order, until one successfully accepts or permanently rejects the mail.
2. If you wish the routing server(s) to be treated as MX records, set Treat route entries as MX records to Yes. Otherwise, leave it at No.
Note: You should normally not treat your route entries as MX records. Unless you know for sure that they specify correct MX records that will route your mail correctly, setting this setting to Yes could cause mail loops. If you use IP addresses rather than host names for your routes, you must not set Treat route entries as MX records to Yes.
3. If you wish to route mail to a non-standard port (normally, SMTP traffic goes to TCP port 25), enter the port number in the Destination TCP port box. Note that only the CanIt-Domain-PRO site administrator can specify a port that is less than 1024 and that is not 25 or 587.
Note: CanIt-Domain-PRO imposes a system-wide limit of 12 different non-standard TCP ports. This limit is caused by technical limitations in Sendmail and cannot be raised. Again due to Sendmail technical limitations, if you specify more than one server in the Route To list, all servers must listen on the same port.
Note: If you use a non-standard port for mail routing and are using the Verification Servers feature to validate recipients, you may need to specify the same non-standard port in Setup : Verification Servers.
CanIt-Domain-PRO — AppRiver, LLC 62 CHAPTER 5. CANIT-DOMAIN-PRO SETUP
4. CanIt-Domain-PRO can send an alert when either the number of queued messages or the age of the oldest queued message exceeds a threshold. The Number of queued messages required to trigger notification and Age of queued message in hours required to trigger notification settings control when warnings are sent. In order to have alerts generated, you must enter an email address in the Notification Email Address field, and this address cannot be in the same domain as the domain being routed. (If mail for example.org is backing up in the queue, it is probably pointless to attempt to mail an alert to someone in that domain.) If you wish to send an alert to more than one recipient address, enter a comma-separated list of email addresses. For example: [email protected], [email protected]
5. Click Submit Changes.
5.5.1 Outbound Relaying
Normally, CanIt-Domain-PRO refuses to relay mail for domains that do not appear in the Domain Routing Screen. However, if you wish to relay outbound mail through CanIt-Domain-PRO, you can specify networks for which relaying should be enabled. To do this:
1. Click on Setup and then Known Networks
2. Enter the network from which relaying should be allowed. For example, to allow all machines on the Class C network 192.168.2.0 to relay outbound mail, enter 192.168.2.0/24
3. Enable the Allow Relaying checkbox.
4. Click on Submit Changes.
5.5.2 Outbound Relaying for Select Domains
Normally, CanIt-Domain-PRO enables the Relay Unlisted Domains (rud) flag for a Known Network. This means that if Allow Relaying is enabled, then mail from the given network is relayed regardless of the sending domain. If you wish to relay mail from a network only for specific domains, perform the following steps:
1. Click on the Show button in the Associated Domains column corresponding to the appropriate Known Network.
2. Enter a domain in the “Add Domain” text box and click Submit Changes
3. Repeat the previous step for all sending domains that should be relayed from the given network.
4. Disable the Relay Unlisted Domains flag.
CanIt-Domain-PRO — AppRiver, LLC 5.6. CLUSTER MANAGEMENT 63
5.6 Cluster Management
The CanIt-Domain-PRO Web interface has a page for managing your CanIt Cluster. To access the page, click on Setup and then Cluster Management. The Cluster Management page appears:
Figure 5.8: Cluster Management Page
The various machines in your cluster are shown. Each member of a CanIt-Domain-PRO cluster can run one or more services. The services are:
• Scanner – this service scans mail flowing through the cluster member. Typically, all members of a CanIt-Domain-PRO cluster will run this service, although large installations may not run a scanner on the database host. NOTE: All nodes should be marked “Scanner” even if they don’t actually act as MX hosts. This is to permit locally-generated traffic such as cron messages to be delivered. Do not turn off the “Scanner” service on any cluster members. If you think you need to, please contact AppRiver support first.
• Ticker – this service runs periodic maintenance jobs. Exactly one host in the cluster must run this service. That host must also run the Scanner service.
• Main Database – this service is the main PostgreSQL database. One host in the cluster must be an active database server, but it is possible to set up a failover database server.
• Web Server – this service provides the Web interface and REST-based API. it can run on as many hosts as you like.
• Inbound – this host processes inbound email. Always leave this setting enabled; if you think you need to disable it, please contact AppRiver technical support. Note that the ticker must be marked as an Inbound scanner. If a host is not marked as inbound, then:
1. Verification Server checks are skipped. 2. (Appliance Only) Domain Routing entries are ignored by the host (mail is routed solely according to MX records) and Sendmail access entries are not created for domains in the Domain Routing table.
• Outbound – this host processes outbound email. Always leave this setting enabled; if you think you need to disable it, please contact AppRiver technical support. If a host is not marked as outbound, then:
1. The “Allow Relaying” Known Networks flag is ignored.
CanIt-Domain-PRO — AppRiver, LLC 64 CHAPTER 5. CANIT-DOMAIN-PRO SETUP
2. The “Force to Stream” Known Networks entry is ignored.
• Sync Bayes – this host requires Bayes data for processing email. Always leave this setting enabled; if you think you need to disable it, please contact AppRiver technical support.
• Log Host – this host contains mail logs that should be indexed for the Log Search Feature. Note that this feature is available only on Hosted CanIt and our CanIt appliances.
• Storage Manager – if you are using the Storage Manager, the table will indicate on which hosts it is running. You can run Storage Manager on as many hosts as you like.
5.6.1 Bandwidth Optimization for Copying Files
CanIt-Domain-PRO copies files from the ticker to other cluster members on a regular basis. For example, this is how Bayes databases are distributed. If every cluster member is given a non-blank location, then CanIt-Domain-PRO can optimize the use of relatively slow links. Here is an example: Suppose you have three data centres A, B and C. Suppose that within a data centre, cluster members are connected by 1Gb/s Ethernet, but between data centres there is only a 10 Mb/s link. Furthermore, suppose that you have three hosts in each data centre with the ticker host in A. If you set the locations of the hosts to “A”, “B”, and “C” according to which data centre they are in, then when CanIt-Domain-PRO copies files, it performs the following steps:
1. The ticker copies the files to all machines in its location (A) and to one machine into each of the other locations. These other machines are called the representatives.
2. Then for each representative, CanIt-Domain-PRO copies the files from that machine to the other machines that are in the same location as the representative.
You can use whatever labels you like for the Location field as long as machines that are in the same location have the same label. Note also that every machine in the cluster must have a non-blank location or CanIt-Domain-PRO will not perform bandwidth optimization.
5.6.2 Altering Services on a Cluster Member
To alter the services running on a cluster member:
1. Check or uncheck the appropriate checkbuttons or radio buttons in the Scanner, Ticker, etc. columns. Note that that the Database and Web Server checkboxes are informational; changing them won’t actually change which services run on the host. And the Storage Manager column is read-only because storage manager hosts are configured in the Storage Manager Wizard.)
2. Click Submit.
CanIt-Domain-PRO — AppRiver, LLC 5.7. KNOWN NETWORKS 65
5.6.3 Renaming of Cluster Members
If you rename a CanIt-Domain-PRO host, the cluster management software usually picks up on the name change automatically. If, however, nonexistent or dead hosts appear in the Cluster Management table, you can delete them. To delete hosts:
1. Enable the appropriate checkboxes in the Delete column.
2. Click Submit.
Internally, CanIt-Domain-PRO identifies hosts with a UUID, which is an identifier that looks some- thing like this: 30829e66-4df8-11e2-95d2-e6dca73e5dae The UUID of a given CanIt-Domain-PRO host is stored in the file /etc/mail/canit/canit-cluster-member-id. You can find the UUID by running this command: # head -n 1 /etc/mail/canit/canit-cluster-member-id In the Cluster Management screen, hovering over the host name reveals the UUID of the host. This can help you to decide which to delete in case two identical host names appear.
5.7 Known Networks
Note: This section describes features that only the CanIt-Domain-PRO System Administrator can use. CanIt-Domain-PRO allows you to enter a list of “known networks”. These are typically networks that you control, and for which you wish to alter the normal CanIt-Domain-PRO processing flow. For example, you may not wish to scan outgoing mail for spam; if all outgoing mail originates from a known set of IP addresses, you can tell CanIt-Domain-PRO to skip spam-scanning for mail originating from those IP addresses. To edit the list of known networks, click on Setup and then Known Networks. The Known Networks page appears:
Figure 5.9: Known Networks
Each network appears as a row in the table. By default, CanIt-Domain-PRO abbreviates the attribute names to avoid a very wide page that requires horizontal scrolling. You can hover over the abbreviation to see the full attribute name, or click Full Headings to show the full attribute names. In the example in Figure 5.9:
CanIt-Domain-PRO — AppRiver, LLC 66 CHAPTER 5. CANIT-DOMAIN-PRO SETUP
• The host 192.168.10.6 will not be looked up in any RBL blocklists.
• Mail originating from 192.168.10.6 will not be scanned for spam:
• Mail originating from 192.168.10.6 cannot be blocked. That is, any sender, domain or host block rules will be ignored.
• Greylisting will be turned off for 192.168.10.6.
• 192.168.10.6 will never be banned by the Dictionary Attack Detector.
• Mail originating from 192.168.10.6 will be streamed into the Outgoing stream, no matter what.
To add a network to the list of known networks:
1. Enter the network address in the Network box. A network address can either be a single IP address, or a network address in CIDR notation: a.b.c.d/bits. In this notation, a through d are decimal numbers from 0 to 255, and bits is a number from 1 to 32 specifying how many bits of the address are significant. Note that the remaining bits (32 – bits) must be zero. (For more information on CIDR notation, please see http://en.wikipedia.org/ wiki/Classless_Inter-Domain_Routing.) Here are examples of network addresses:
• 192.168.1.0/24 specifies the Class C network 192.168.1.0 through 192.168.1.255. • 10.5.2.0/23 specifies the IP addresses 10.5.2.0 through 10.5.3.255. • 192.168.5.5/24 is invalid, because the lower 8 bits of the address must be zero.
2. Choose the characteristics you wish to apply to hosts in the known network (you may need to click on Full Headings to see the full names of each characteristic.)
• To skip DNS-based RBL lookups, enable Skip RBL Lookups (srl). • To skip spam-scanning, enable Skip Spam Scan (sss). • To skip virus-scanning, enable Skip Virus Scan (svs). • To skip filename and filename extension checking, enable Skip Extension Rules (ser). • To skip MIME-type checking, enable Skip MIME-Type Rules (smr). • To skip enforcement by CanIt-Domain-PRO of maximum message size, enable Skip Size Limit Checks (ssl). • To prevent sender, domain or host block rules from applying to mail sent from the network, enable Prohibit Block Rules (pb). • To skip greylisting for hosts in the network, enable Skip Greylisting (sg). • To skip SPF checks for hosts in the network, enable Skip SPF Checks (ssc). Note that this also disables DKIM and DMARC checking. • To disable delay rules for hosts in the network, enable Skip Delay Rules (sdr). • To disable attachment-stripping rules for hosts in the network, enable Skip Attachment Stripping (sas).
CanIt-Domain-PRO — AppRiver, LLC 5.7. KNOWN NETWORKS 67
• To prevent any hosts in the network from being banned by the Dictionary Attack Detector (Section 5.3.3), enable Omit from Dictionary Attack Detection (oda). • If all hosts in the network are “friendly”, then enable Friendly Host (fh). If mail from a friendly host must be rejected, then CanIt-Domain-PRO simply discards it rather than replying with an SMTP 5xx code. This is used to prevent backscatter. • To have CanIt-Domain-PRO parse Received: headers to find the sending relay, enable Parse Received Headers (prh). CanIt-Domain-PRO parses through the headers until it finds a host that isn’t in a known-network with this flag set. If CanIt-Domain-PRO parses the Received: headers, then the host that directly initiated the SMTP connection to the CanIt-Domain-PRO scanner is called the Connecting Relay whereas the host parsed out of the Received: headers is called the Sending Relay. If CanIt-Domain-PRO does not parse the Received: headers, then the Sending Relay and the Connecting Relay are one and the same. • To auto-allow-always recipients of messages from a known network, enable Auto-Allow Recipients (aar). This means that for messages originating from the network, the recipi- ents of the message are allowed-always in the Sender Rule table. Note that auto-allowing is not applied if any of these conditions holds: – There is already a sender rule for the recipient in the stream in which the Sender Allow rule would normally be created. – The message has a “Precedence: bulk” or “Precedence: junk” header. – The message has an “Auto-Submitted” header, as specified in RFC 3834. – The message is a bounce message (in other words, the sender is <>. – The message subject contains “[no-whitelist]”. In this case, the [no-whitelist] tag is removed before the message is delivered (so that the recipients do not see it.) – The message subject matches the regular expression ˆout of.*office case- insensitively. – Auto-allowing has been disabled under Preferences : Quarantine Settings for the sender’s stream. Note that some auto-responder software ignores RFC 3834 and fails to add an “Auto- Submitted” header. This could lead to situations in which CanIt-Domain-PRO auto-allows someone because of an auto-response. If you cannot convince your auto-responder soft- ware to add an Auto-Submitted header, you should complain to the vendor of that software in an attempt to make it RFC-compliant. If a stream inherits from a final stream, then the allow-always rule is created in the final stream. Otherwise, it is created in the actual stream itself. Please see Section 10.3.1 on page 175 for the precise definition of a final stream. • To allow outbound mail from the network to be relayed through the CanIt-Domain-PRO machine, enable Allow Relaying (ar). Note: Outbound relaying can be enabled from the Web interface only on CanIt-Domain-PRO appliances or Linux-based RPM builds with the appliance RPMs installed. Also, this flag is ignored on nodes that are not marked “Outbound” in the Cluster Members Table.
CanIt-Domain-PRO — AppRiver, LLC 68 CHAPTER 5. CANIT-DOMAIN-PRO SETUP
• To rate-limit how many recipients per hour a given sender can send to, enter a number in Per-Sender Recipient Rate Limit. If you use this option, you must also enter a Force To Stream value. Rate-limiting is described more fully in Section 5.8. • If you wish to rate-limit by sending IP address as well as sending email address, enter the appropriate limit in the Per-IP Recipient Rate Limit box. See Section 5.8.1 for details. • To force all mail from the network to be streamed into a specific stream, enter the name of the stream in the Force To Stream box. Note: You must supply a fully-qualified stream name of the form realm:stream. If you use the magic value @@ as the realm name, then the message is forced into the realm of the envelope sender and the given stream. For example, if you set the Force To Stream value to @@:outgoing and the domain example.com is mapped to the realm example-com, then mail from [email protected] originating from the known network will be forced into the stream outgoing in the realm example-com. Alternatively, you can use a forced-to stream name of the form somerealm:@@. Continuing our example, that would force mail from [email protected] originating from the known network into the stream example-com in the realm somerealm. Note that the Force To Stream box is ignored on nodes that are not marked “Outbound” in the Cluster Members Table. If mail is forced to a stream, CanIt-Domain-PRO does not perform any SPF, DKIM or DMARC checks, since forcing mail to a stream typically indicates outbound mail.
3. Click Submit Changes to have your changes take effect.
To edit an existing known network, simply adjust the attributes as required and click Submit Changes. To delete a known network, enable the Delete? checkbox and click Submit Changes.
5.7.1 Associating Domains with Known Networks
Each Known Network may be associated with any number of domains. To view the list of associated domains for a given network, click on the “Show” button in the “Associate Domains” column. The list of Associated Domains appears:
Figure 5.10: Known Network with Associated Domains
CanIt-Domain-PRO — AppRiver, LLC 5.8. RATE-LIMITING OUTBOUND MAIL 69
In this example, the domains example.com and example2.net are associated with the network 192.168.7.88. Additionally, email originating from that network is normally forced into the out- bound stream, but if email originating from that network has an envelope sender whose domain is example2.net, then it will be forced into the outbound-example2.net To associate a domain with a network, simply enter the new domain name in the Domain box. You may optionally specify a domain-specific Force To Stream value in the Force To Stream box; this overrides the general Force To Stream setting associated with the network. Click Submit Changes to add the domain. To remove domains from the list of associated domains, enable the appropriate checkboxes in the Delete? column and click Submit Changes. If you have enabled the Allow Relaying (ar) flag on a known network, the Relay Unlisted Domains (rud) flag will normally be on as well. This tells CanIt-Domain-PRO to relay all mail from the specified network, regardless of the sender domain. If, however, you turn off the Relay Unlisted Domains (rud) flag, then CanIt-Domain-PRO will refuse to relay mail from the given network unless the domain of the envelope sender is in the list of domains associated with the network. (CanIt- Domain-PRO always permits relaying of the null return path, <>.) We do not recommend turning off Relay Unlisted Domains unless you are absolutely sure the network never originates or forwards mail from a domain not in the list of associated domains.
5.7.2 Overlapping Networks
If you add two networks that overlap, CanIt-Domain-PRO will use the most-specific network for a given host. That is, CanIt-Domain-PRO will choose the smallest network that con- tains a given host. For example, if you create the known networks 192.168.1.0/24 and 192.168.1.240/28, then hosts in the range 192.168.1.240 through 192.168.1.255 will use the 192.168.1.240/28 settings, whereas hosts from 192.168.1.0 through 192.168.1.239 will use the 192.168.1.0/24 settings.
Note: Because of how Sendmail’s access map works, the handling of overlapping networks described above does not apply to the Allow Relaying (ar) setting. Instead, relaying will be permitted for any host in a network with the flag enabled even if there is a more-specific network with the flag turned off. If this is of concern, then you need to split your Known Networks entries into non-overlapping networks.
5.7.3 The SMTP-AUTH Pseudo-Network
CanIt-Domain-PRO supports a pseudo-network called SMTP-AUTH. (It must be entered exactly like that in upper-case.) Any Known Network settings for this network will be applied to users who authenticate using SMTP AUTH. This lets you do things like force authenticated mail into a particular stream or skip spam-scanning for authenticated users.
5.8 Rate-Limiting Outbound Mail
Note: This section describes features that only the CanIt-Domain-PRO System Administrator can use.
CanIt-Domain-PRO — AppRiver, LLC 70 CHAPTER 5. CANIT-DOMAIN-PRO SETUP
The Known Networks feature allows you to limit the number of recipients a given sender can mail in an hour. This can be useful to catch compromised internal hosts that are used to send spam. Here is how rate-limiting works:
• Normally, you can only rate-limit mail from a Known Network. This is because rate-limiting is designed to rate-limit outbound mail from a set of machines under your control. Under special circumstances, you can enable rate-limiting for any stream other than default, but you should normally not use rate-limiting for inbound email.
• To specify a rate limit, enter the maximum number of recipients per hour that a given sender can send to. A reasonable value might be 500 to 1000; a value of 0 disables rate-limiting completely. (Enter the value in the Recipient Rate Limit column of Known Networks.)
• The rate limit may be positive or negative. CanIt-Domain-PRO treats limits as follows:
– If the limit is positive, then a sender who exceeds the limit is permanently blocked. Any mail from that sender is rejected with an SMTP permanent-failure code. – If the limit is negative, then a sender who exceeds the absolute value of the limit is temp- failed. Any mail from that sender is rejected with an SMTP temporary-failure code.
• You must also specify a Force To Stream value in order to use rate-limiting.
If a sender exceeds the rate limit, CanIt-Domain-PRO creates a Sender rule in the Force To Stream stream. The rule rejects all mail from the sender. This has the effect of completely disabling all outbound mail from the sender address. The sender rule that CanIt-Domain-PRO creates is set to expire automatically three days after it is created. CanIt-Domain-PRO also sends an email to the CanIt-Domain-PRO administrator informing him or her of the rule that blocks the sender. Note that the sender will be unable to send outbound mail until the administrator goes into the Force To Stream stream and manually removes the rule that blocks the sender (or until the rule expires after three days.)
Note: Any sender that has any Sender Rule defined in the outbound stream will not be subject to rate- limiting. You can use this as an “escape hatch” to permit certain senders to send high volumes of mail; simply always allow those senders in the forced-to stream (or add a “Hold if looks like spam” rule for those senders.) However, you should be very careful to do this only for legitimate senders who are unlikely to have their accounts hijacked. Also, note that if a sender is allowed for any reason (ie, a sender allow rule, domain allow rule or host allow rule), rate-limiting will not apply. For this reason, you should be very judicious about the allow rules you create in the forced-to stream and consider setting up the forced-to stream not to inherit from the default stream.
Note: If you enable rate-limiting on a Known Network, be sure that you do not enable the “Prohibit Block Rules” option for that network. Otherwise, rate-limiting rules will be ignored! In addition, if you rate-limit the SMTP-AUTH pseudo-network, be sure not to enable the global setting “Always allow users who use SMTP authentication” (G-3600) or rate-limiting will be ignored.
CanIt-Domain-PRO — AppRiver, LLC 5.8. RATE-LIMITING OUTBOUND MAIL 71
5.8.1 Rate-Limiting by IP Address
Normally, CanIt-Domain-PRO applies rate-limiting on a per-sender email address basis. If you enable the Per-IP Recipient Rate Limit feature in Known Networks, CanIt-Domain-PRO will also apply rate-limiting to the sending IP address. If the Known Networks entry has Parse Received Headers enabled, then the IP address that is rate-limited is extracted from the Received: headers. As with the sender rate-limit, the IP-based rate limit may be positive or negative, with positive limits yielding an SMTP permanent-failure code and negative ones yielding a temporary-failure code if the limit is exceeded.
Note: Be very careful when enabling IP-based rate-limiting. If all of your mail goes out through one server and you accidentally turn on rate-limiting by IP address without enabling Received: header parsing, you may end up blocking all outbound mail. The rule of thumb is as follows:
• If various clients connect directly to the CanIt-Domain-PRO server to send outbound email, you must not enable Parse Received Headers on the Known Network containing the client IP addresses.
• If clients relay via an SMTP server that subsequently relays out via the CanIt-Domain-PRO server, then you must enable Parse Received Headers.
5.8.2 Fine-Grained Rate-Limiting Rules
Note: By default, realm administrators do not have permission to create rate-limiting rules, but permission can be granted by the CanIt-Domain-PRO site administrator. In addition to per-known-network rate-limits, you can create finer-grained rate-limiting rules by click- ing Rules : Rate Limiting. The Rate-Limiting Rules page appears (Figure 5.11):
Figure 5.11: Rate-Limiting Rules
Normally, CanIt-Domain-PRO only applies rate-limiting rules for a stream if the mail has been forced into that stream by a Known Networks match. However, for streams other than default you can
CanIt-Domain-PRO — AppRiver, LLC 72 CHAPTER 5. CANIT-DOMAIN-PRO SETUP
change the setting “Apply rate-limiting rules in stream?” to Always to always apply the fine-grained rate-limiting rules, even if mail was not forced into the stream by a Known Networks match. We do not recommend applying rate-limiting to inbound streams; you should normally never change this setting. Rate-limiting rules permit you to use one of the following in the Originator column: For sending email addresses:
• A full email address, which applies to a specific sender.
• A domain name, which applies to all senders in that domain. Note that a full email address rule will override a domain rule.
• A single asterisk, which applies to senders that don’t have a full email address or a domain name match.
For sending domains:
• A domain name prefixed by ‘@’ which limits mail from all senders within that domain.
• The value @* which applies to all domains.
The difference between a sending email address limit and a domain limit is that domain limits apply cumulatively to any email address within the domain. Thus, a limit of 100 recipients per hour for example.com limits any given sender within the “example.com” domain to 100 recipients per hour. On the other hand, a limit of 100 recipients per hour for @example.com limits the total number of recipients for all addresses within the “example.com” domain to 100 recipients per hour. For sending machines:
• An IPv4 or IPv6 address, which applies IP-based rate-limiting to a specific IP address.
• The IP address 0.0.0.0, which applies IP-based rate-limiting to machines that don’t have a spe- cific IP address rule. (This includes IPv6 machines.)
In the example in Figure 5.11, the following sender rate limits apply:
• The sender “[email protected]” is limited to 100 recipients per hour.
• The sender “[email protected]” is limited to 500 recipients per hour.
• The sender “[email protected]” has no rate-limits set.
• All senders in the “example.com” domain are limited to 200 recipients per hour.
• All other senders are limited to 150 recipients per hour by the “*” entry.
The following domain-based rate limits apply:
CanIt-Domain-PRO — AppRiver, LLC 5.8. RATE-LIMITING OUTBOUND MAIL 73
• Senders in the domain “example.net” are cumulatively limited to 300 recipients per hour.
• Senders in all other domains are cumulatively limited to 200 recipients per hour.
And finally, the following IP-based rate-limits apply:
• The machine 10.2.3.4 is allowed to send to 10000 recipients per hour.
• All other machines are limited to 500 recipients per hour by the “0.0.0.0” entry.
To create a rate-limiting rule:
1. Enter the sender address, domain name, IP address, “*” or “0.0.0.0” in the Originator box.
2. Enter a number from 0 to 100000 in the Hourly Limit box. An entry of 0 means that no rate-limiting is to be applied. Any other entry N applies a rate-limit of N recipients per hour.
3. Select an action from the Action pull-down. Available actions are:
• Reject — if the rate-limit is exceeded, CanIt-Domain-PRO creates a rule that blocks the sender or IP address. Mail from the blocked originator will simply be rejected. • Tempfail — if the rate-limit is exceeded, CanIt-Domain-PRO creates a rule that always tempfails the originator. This permits administrators to examine the situation and unblock the originator if necessary. • Hold Always — if the rate-limit is exceeded, CanIt-Domain-PRO quarantines all mail from the originator. Again, this permits administrators to examine the situation and release the quarantined messages if they are legitimate.
4. Enter a number from 1 to 30 in the Block Duration field. When CanIt-Domain-PRO creates a Reject, Tempfail or Hold Always rule, it sets it up to expire after N days, where N is the number you enter for Block Duration.
5. If you wish, enter a comment in the Comment box to help remind you why you made the rule.
6. Click Submit Changes
To delete rate-limiting rules, enable the appropriate checkbox in the Delete? column and click Submit Changes.
5.8.3 Notes about Rate-Limiting Rules
• Rate-limiting rules are applied only for mail that is forced into a stream by a Known Networks entry. Normal inbound mail is never rate-limited.
• Rate-limit settings are inherited across streams. CanIt-Domain-PRO uses the best match in the most-specific stream to determine the rate-limit. For example, suppose the stream outbound inherits from the stream default. Suppose that outbound has a rule for “example.com” and that default has rules for “[email protected]” and “*”. Then:
CanIt-Domain-PRO — AppRiver, LLC 74 CHAPTER 5. CANIT-DOMAIN-PRO SETUP
1. An originator “[email protected]” will use the “example.com” entry from outbound. That’s because outbound is more specific than default and it did have an entry that matched the originator. 2. An originator “[email protected]” will use the “*” entry from default because no rule in outbound matched.
• When a rate-limit is hit and a rule is created, the rule is always created in the forced-to stream from the Known Networks entry. Additionally, CanIt-Domain-PRO sends an email to the site administrator informing him or her that the originator has exceeded the rate limit.
• If you use a Hold Always rule, make sure the forced-to stream is not a tag-only stream. Other- wise, mail from the originator will be tagged rather than quarantined.
• Make sure the forced-to stream is not opted-out of spam-scanning or any hold, tempfail or reject rules will be ignored.
• If an originator does not match any rate-limiting rules, then the rate limits from the Known Network entry (if any) apply.
• The “Hourly Limit” refers to the total number of recipients mailed, not the number of unique recipients. For example, if a given sender sends 50 copies of a message all to the same recipient, that counts as 50 recipients, not one recipient.
5.9 Features
Note: This section describes features that only the CanIt-Domain-PRO System Administrator can use. The Features page allows you to globally disable certain CanIt-Domain-PRO features to reduce the number of database queries. Note that disabling a feature completely disables it system-wide. Unless you know for sure that you don’t need a feature, and you know that the load savings will be worth turning it off, you should leave all features in their default states. To disable a set of features, click on No in the Enabled column for the features you want to disable. Then click Submit Changes. Some features are disabled by default because they are considered dangerous or are only useful in special situations. You can enable such features by selecting Yes in the Enabled column and then clicking Submit Changes.
5.9.1 Direct Queue Injection
Normally, when CanIt-Domain-PRO needs to split an incoming message destined for several streams into several single-stream messages, it performs the following actions:
1. It remails a copy of the message for each stream by invoking sendmail with appropriate arguments.
2. It discards the original message.
CanIt-Domain-PRO — AppRiver, LLC 5.10. SYSTEM CHECK 75
Remailing a message with Sendmail is expensive because multiple copies of the message data are made and Sendmail uses expensive disk synchronization operations after each copy. CanIt-Domain-PRO can instead directly inject copies of the streamed messages into Sendmail’s local client queue. This saves disk I/O because only one expensive synchronization operation is needed (not one per copy.) Also, the data can be hard-linked instead of copied, saving disk space. In order for this to work, the defang user must be a member of the smmsp group. (This is the case if you are running an appliance or an RPM build.) Additionally, you must enable the “Insert Streamed Mail Directly Into Sendmail Queue” feature under Setup : Features.
5.10 System Check
The System Check page runs some sanity checks on your CanIt-Domain-PRO installation. It also displays the current versions of RPTN data and AppRiver rule sets. A typical System Check page is shown in Figure 5.12:
CanIt-Domain-PRO — AppRiver, LLC 76 CHAPTER 5. CANIT-DOMAIN-PRO SETUP
Figure 5.12: System Check
In addition to running a few local tests, viewing the System Check page also shows the results of cluster-wide tests performed on a periodic basis. If System Check indicates a problem, you should take action to fix it immediately. The various System Check tests are outlined in AppendixL.
5.11 Templates
CanIt-Domain-PRO uses templates to configure how Bayes training information is added to messages and to configure the appearance of Pending Message Notifications. These templates may be configured on a per-realm basis. To configure templates, click on Setup and then Templates. The Templates screen appears:
CanIt-Domain-PRO — AppRiver, LLC 5.11. TEMPLATES 77
Figure 5.13: Templates
The various templates you can configure are:
• Base URL of CanIt installation is used to construct URLs in messages sent out by CanIt- Domain-PRO.
• Base URL for URL-Rewriting is used to construct URLs when rewriting URLs (Chapter 14). Normally, you should leave this template blank, in which case the Base URL of CanIt instal- lation is used.
• E-Mail address of CanIt System Administrator is the e-mail address to which CanIt-Domain- PRO sends certain warning messages or alerts.
• Source E-Mail address of CanIt notifications is the sender address used by CanIt-Domain- PRO when it e-mails notifications. This is the envelope sender address.
• Full name for sender of CanIt notifications is the full name placed in the From: header of CanIt-Domain-PRO notifications.
• Header From: address of sender of CanIt notifications is the email address placed in the From: header of CanIt-Domain-PRO notifications. If this template is left blank, then CanIt- Domain-PRO uses the value from “Source E-Mail address of CanIt notifications”.
• SMTP reply for a rejected incident is the text returned with the SMTP permanent failure code when CanIt-Domain-PRO rejects an incident.
• SMTP reply for a blocked entry is the text returned with the SMTP permanent failure code when CanIt-Domain-PRO rejects a host, sender or domain that is blocked.
CanIt-Domain-PRO — AppRiver, LLC 78 CHAPTER 5. CANIT-DOMAIN-PRO SETUP
• Header note for an always-allowed entry is the note CanIt-Domain-PRO places in the X- Spam-Score header when a host, sender or domain has an always-allow rule.
• Plain-text training link body specifies the appearance of Bayesian training links added to plain-text messages.
• HTML training link body specifies the appearance of Bayesian training links added to HTML messages.
• Pending notification e-mail subject specifies the subject to put in Pending Notification mes- sages.
• Pending notification e-mail body specifies the body of Pending Notification messages
• Preamble before notification details specifies the preamble before the detailed list of held messages (for users who select verbose notifications.)
• Detailed pending notification entry specifies the format for each held message in detailed notifications.
• Subject for Add Alternate Address e-mail specifies the subject of the confirmation e-mail sent when someone attempts to add an Alternate Address to his/her stream.
• Body for Add Alternate Address e-mail is the body of the confirmation e-mail described above.
• Header for ’Webform’-style Pending Notification is the HTML preamble used for “Web- form” pending notifications.
• Footer for ’Webform’-style Pending Notification is the HTML postamble used for “Web- form” pending notifications.
• Subject line for Periodic Reports is the subject used by CanIt-Domain-PRO when mailing out periodic reports.
• Body of Periodic Report e-mail is the body used by CanIt-Domain-PRO when mailing out periodic reports. It should consist of valid HTML.
• Text boilerplate when attachments are stripped is appended to the first text/plain email part if an attachment is stripped and stored on the CanIt-Domain-PRO server.
• HTML boilerplate when attachments are stripped is appended to the first text/html email part if an attachment is stripped and stored on the CanIt-Domain-PRO server.
• Text boilerplate when attachments are discarded is appended to the first text/plain email part if an attachment is stripped and discarded.
• HTML boilerplate when attachments are discarded is appended to the first text/html email part if an attachment is stripped and discarded.
• Forgot-your-Password Link or Text is the link or text used for the Forgot your Password? message on the login page.
CanIt-Domain-PRO — AppRiver, LLC 5.12. THEME CUSTOMIZATION AND BRANDING 79
• HTML content for anti-phishing URL Redirection page is the content of the URL Proxy warning message. See Chapter 14, “URL Proxying”, for details on the URL Proxying feature. • HTML content for anti-phishing URL Redirection page encountering a Phishing URL is the content of the URL Proxy message when a suspected phishing link is encountered. See Chapter 14, “URL Proxying”, for details on the URL Proxying feature.
Note that many templates include various “replacement tags”. For example, in the training link tem- plates, the sequence of characters %spamurl or %{spamurl} will be replaced with a URL that votes the message as spam. To see the list of available replacement tags, click on the “(Tags)” link near the template entry box. If you change the value of a template in a non-base realm, you can revert to the previous value by clicking the “(Revert to Original)” link next to the template name.
5.12 Theme Customization and Branding
CanIt-Domain-PRO ships with several themes which control the “look and feel” of the Web interface. Some of those themes can be customized. That is, although the basic layout of the theme cannot be changed via the web interface, the colors of various elements can be and (in some cases) the logo can be changed as well. This permits you to “brand” CanIt-Domain-PRO with your corporate logo. To customize a theme, click on Setup : Theme Customization. The Theme Customizations page appears:
Figure 5.14: Theme Customizations
Note: The list of available customizations is specific to the current theme and realm. If you switch themes or realms, then the list of available customizations will change. Also, some elements such as images on the login page may only be cusomizable in the base realm and therefore can be customized only by the site administrator. To activate a customization, enable the corresponding Active radio button and click Submit Changes. That customization will become active for the current theme and realm. It will also be active for all subrealms unless overridden within a subrealm. To deactivate all customizations, click Deactivate All. This will revert the current theme and realm including subrealms to the default un-customized appearance.
CanIt-Domain-PRO — AppRiver, LLC 80 CHAPTER 5. CANIT-DOMAIN-PRO SETUP
To delete a customization, enable the corresponding Delete? checkbox and click Submit Changes.
5.12.1 Creating or Editing a Customization
To add a new customization, click Add New Customization. To edit an existing customization, click on the name of the customization you wish to edit. In either case, the Theme Customization Editor appears:
Figure 5.15: Theme Customization Editor
The Theme Customization Editor lets you alter the appearance of various components of the web page. To edit a customization:
1. If you are adding a new customization, the Customization Name field will be blank. Enter the name of your new customization. Note that customization names must be unique for a given theme and realm.
2. To change image items, upload a GIF, JPEG or PNG file from your computer.
3. To change color items, enter a “#” followed by an HTML color triplet in the text box. If you click on the color swatch to the right of the text box, you can pick a color from a color selector.
4. Some themes may allow you to enter arbitrary CSS information. This lets you have very fine control over the appearance of the theme, but you should not make use of this facility unless you are very familiar with HTML and CSS.
5. If you want to revert a particular item to its theme default, enable the Revert to default? check- box.
6. Click Save to save your customization and continue editing it. Or click Save and return to list to save your customization and return to the list of available customizations.
Note that while you are editing a customization, it becomes active so you can see in real-time what the customized theme looks like. Other users, however, will not see the customized theme until you activate it from the list of customizations.
CanIt-Domain-PRO — AppRiver, LLC 5.13. HTTPS 81
5.12.2 Emergency Recovery from Bad Theme Customization
If you make a mistake while creating a theme customization and end up with web pages you can’t read or navigate, follow these emergency instructions:
• Look at the URL in the URL bar of your browser. If it contains a question-mark, add the following text on the end of the URL: &disable theme customization=1 If it does not contain a question mark, add this at the end: ?disable theme customization=1
• Press Enter to visit the newly-edited URL
• Navigate back to Setup : Theme Customization and fix the problem. Note that you have to adjust the URL in the URL bar each time you navigate to a new page, so you might need to do it a few times until the problem is fixed.
5.13 HTTPS
Note: This feature is available only on Debian-based Appliances. On CanIt-Domain-PRO appliances, HTTPS is enabled by default, but with dummy self-signed certifi- cates. If you would like to install your own certificates, click on Setup : HTTPS. Then:
1. Copy-and-paste your SSL certificate into the first text box. If your certificate provider requires you to install an intermediate certificate chain, paste the entire contents of the certificate chain file into the first text box immediately after you paste in your SSL certificate.
2. Copy-and-paste the corresponding server key into the second text box. The server key must not be encrypted or the Web server on the appliance will fail to start.
3. Click Submit Changes to install the key and certificate.
5.14 The Domain Mapping Table
Recall from Figure 2.4 on page 34 that CanIt-Domain-PRO uses a Domain Mapping Table to deter- mine how to stream messages for each domain. The table contains a list of domains with a correspond- ing lookup method. To edit the Domain Mapping Table, click on Setup and then Domain Mappings. The Domain Mappings page appears:
CanIt-Domain-PRO — AppRiver, LLC 82 CHAPTER 5. CANIT-DOMAIN-PRO SETUP
Figure 5.16: Domain Mappings
To add a mapping method for a particular domain, enter the domain name in the top row of the table and select a value in the Mapping column. The possible choices are:
• Database—CanIt-Domain-PRO will look up a stream mapping in the Address Mapping Table (Section 5.15). • AsIs—CanIt-Domain-PRO converts an address to a stream by removing any angle-brackets and converting letters to lower-case. • ChopDomain—CanIt-Domain-PRO converts an address to a stream simply by chopping off the @domain.tld part, removing any angle-brackets, and converting to lower-case. • ChopUser—CanIt-Domain-PRO converts an address to a stream simply by chopping off the address@ part, leaving just the domain (without angle-brackets and converted to lower-case.) • Program—CanIt-Domain-PRO converts an address to a stream by executing the account-info program. Please see Section 7.2.5 on page 157 for more details. Note that Program is deprecated; you should create and use a User Lookup method instead. • None—CanIt-Domain-PRO removes the domain from the Domain Mapping Table. • If you have added external User Lookup methods (Chapter7), some of them may appear as additional choices. For example, the LDAP, Rewrite and Program User Lookup methods can convert an address to a stream. If there are any User Lookup methods added to ancestor realms of the current realm, they will appear as additional choices if they are marked as being available for subrealms.
Click Submit Changes to save your changes. To modify the mapping for an existing domain, select a new mapping in the Mapping column and click Submit Changes.
CanIt-Domain-PRO — AppRiver, LLC 5.15. THE ADDRESS MAPPING TABLE 83
Given a domain sub.example.com, CanIt-Domain-PRO looks up entries in the Domain Mapping Table in the following order, stopping at the first one found:
1. sub.example.com
2. example.com
3. com
4. *
The special domain * is used as a last resort if no better match is found. You may enter a mapping for * to set a default mapping. If there is no * entry and a domain is not found in the Domain Mapping Table, then CanIt-Domain-PRO uses a default lookup method of Database. If you enter a string in the “Filter:” box, then CanIt-Domain-PRO limits the display to entries whose Domain or Mapping columns contain that string.
5.15 The Address Mapping Table
CanIt-Domain-PRO uses an Address Mapping Table (Figure 2.4 on page 34) to map e-mail addresses to streams. The Address Mapping Table is used both for hand-entered entries placed there by the CanIt-Domain-PRO administrator, and for caching the results of the Program mapping method.
Note: If there is an exact match for an email address in the Address Mapping Table, then it is always used, overriding any mapping method. To edit the address mapping table, click on Setup and then Address Mappings. The Address Map- pings page will appear:
CanIt-Domain-PRO — AppRiver, LLC 84 CHAPTER 5. CANIT-DOMAIN-PRO SETUP
Figure 5.17: Address Mappings
To add an entry for a new e-mail address, enter the new address in the Address column of the first row, and enter the stream name in the Mapping column. Then click Submit Changes. To edit an existing entry, edit the text in the Mapping column and click Submit Changes. To delete an entry from the table, click the Delete link in the appropriate row. Click on Not Cached to see only non-cached (hand-entered) entries, Cached to see only cached entries, or Any to see all entries in the Address Mapping Table. If you enter a string in the “Filter:” box, then CanIt-Domain-PRO limits the display to entries whose Address or Mapping columns contain that string.
5.15.1 Wild-Card Entries
The address mapping table may contain three types of wildcard entries:
1. The entry user@* is used if CanIt-Domain-PRO is unable to map an address to a stream with an exact match. If you run several domains, but all user-parts are the same, this wildcard can be useful.
2. The entry *@domain.tld is used if the previous wildcard does not match anything. Use this entry to set up a default stream for e-mail to a particular domain.
3. The entry * is used as a last resort if the previous wildcards did not match.
Note: The addresses postmaster, postmaster@localhost and postmaster@machine name are always mapped to the default stream unless you have a specific entry in the Address Mapping
CanIt-Domain-PRO — AppRiver, LLC 5.16. THE DEFAULT STREAM 85
Table for those addresses. That is, for those three specific addresses, CanIt-Domain-PRO will not use wildcard matches or User Lookups to determine the stream. (In the third address, machine name is the name of the host processing the email.)
5.16 The default Stream
CanIt-Domain-PRO has a built-in stream name that is reserved, and which cannot be used for other purposes. This stream is named default, and is used as follows: If CanIt-Domain-PRO is unable to map an address to a stream (for example, if there are no exact or wildcard matches in the database and the Program method fails), the address is mapped to the hard- coded stream default. The CanIt-Domain-PRO administrator should check the default stream from time to time. The default stream also contains allow and block rules and custom rules that all other streams can inherit. The factory default is for all streams to inherit the lists and rules from default, but you can disable this if you wish. List and rule inheritance work as follows for streams that inherit from default:
• Senders, hosts, domains, extension rules and MIME type rules and are first looked up in the stream’s table. If no entry is found, they are looked up in default’s table.
• Custom rules are evaluated first for the given stream, and then for default. Their scores are added together. Note that if the same rule appears in both the stream’s rule set and default’s rule set, it is counted twice.
5.17 Mapping Scenarios
To give a feel for how to use the mapping, we illustrate a few common scenarios.
5.17.1 Central Scanning with Opt-Out
If you run a mail server and wish to centralize spam-scanning, but you have some users who wish to opt out or handle their own spam, you can do it as follows: In the Address Mapping Table, add this catch-all entry: Address Stream * admin This streams most users’ e-mail to the “admin” stream for centralized processing. If user [email protected] does not want his mail examined by the spam control officer, simply add another entry: Address Stream [email protected] joe This streams mail for [email protected] to joe.
CanIt-Domain-PRO — AppRiver, LLC 86 CHAPTER 5. CANIT-DOMAIN-PRO SETUP
5.17.2 Single Domain
If you host a single e-mail domain, and each user’s login name is simply the first part of his/her e-mail address, setting up mappings is easy. In the Domain Mapping Table, add a single entry: Domain Mapping Method * ChopDomain
5.17.3 Single Domain with Aliases and Mailing Lists
Most likely, your scenario is more complex than in Section 5.17.2. You probably host mailing lists, and have aliases. Let’s suppose you host a list called [email protected], which is run by jane, and that your [email protected] is an alias which gets expanded to jim and bob. You can still use the same Domain Mapping as Section 5.17.2. You have two options for handling the mailing list and sales alias:
1. Allow jane to access the tv-list stream, and allow jim and bob (or delegate one of them) to access the sales stream. Jane will have to remember to check the tv-list quarantine as well as her own quarantine, and similarly for Bob and Jim.
2. Add address mappings like this: Address Stream [email protected] jane [email protected] bob Explicit entries in the Address Mapping Table will override even the ChopDomain method. Here, Jane’s quarantine will contain messages both for herself directly and the mailing list she runs. Bob’s quarantine will contain his messages and messages for sales. (Clearly, you’ve delegated spam handling for sales to Bob alone.) (You can, of course, use Method 1 for tv-list and Method 2 for sales. It’s up to you.)
5.18 Pausing Delivery to Selected Domains
Note: This section is applicable only to CanIt appliances or the Hosted CanIt service. CanIt-Domain-PRO permits you to temporarily pause delivery to selected domains. When delivery to a domain is paused, CanIt-Domain-PRO will continue to accept mail for that domain, but will not attempt to deliver it to the back-end server. Instead, it will simply queue the mail. When delivery is resumed, the mail will be delivered out of the queue.
5.18.1 Pausing Delivery
To pause delivery to a domain:
1. Click on Setup and then Paused Delivery.
CanIt-Domain-PRO — AppRiver, LLC 5.19. THE DOMAIN OVERVIEW PAGE 87
2. Enter the domain whose delivery should be paused in the Domain box.
3. Select a Pause Mode. The choices are:
• Delivery and LDAP/Verification: In this mode, CanIt-Domain-PRO will not attempt to connect to any LDAP servers or verification servers. It will accept mail for addresses that are in its cache and will tempfail mail for addresses that have not been verified or found in the LDAP directory recently. • Delivery Only: In this mode, CanIt-Domain-PRO will not attempt to deliver mail, but will still connect to LDAP servers and verification servers as usual.
4. Enter the expiry time in the Expiry (Minutes) box. The site administrator may specify up to one day (1440 minutes) and realm administrators may specify up to four hours (240 minutes).
5. Click Submit Changes
5.18.2 Resuming Delivery
To resume delivery to a domain:
1. Click on Setup and then Paused Delivery.
2. Enable the Delete? checkbox for the domain whose delivery should be resumed.
3. Click Submit Changes.
5.19 The Domain Overview Page
For convenience, CanIt-Domain-PRO allows you to view the most important settings for your domains in one place. To see the overview, click on Setup and then Domain Overview. If you have more than one domain mapped to your realm, click on the domain name whose overview you desire. The Domain Overview Page appears:
Figure 5.18: Domain Overview Page
The Domain Overview page shows some or all of the following information:
• The Verification Server settings for the domain. Click Edit to modify the settings.
CanIt-Domain-PRO — AppRiver, LLC 88 CHAPTER 5. CANIT-DOMAIN-PRO SETUP
• The Domain Routing settings for the domain. Again, click Edit to adjust the settings.
• The Domain Mapping and Authentication Mapping settings for the domain. If the domain uses a User Lookup for mapping or authentication, you can click on the link in the Value column to see the specific user lookup settings.
• Whether or not the domain correctly validates recipients (as checked by the last nightly cron job.)
• Whether or not the domain’s MX records point at the CanIt-Domain-PRO filter (as checked by the last nightly cron job.)
5.20 Autotask® Integration
Note: Autotask integration is available only on Hosted CanIt and our Debian-based CanIt-Domain-PRO appliances. It is not available in the source or RPM versions of CanIt-Domain-PRO. Autotask® is a Professional Services Automation package designed for IT consultants and managed service providers. CanIt-Domain-PRO can interface with Autotask to automatically generate billing information so you can invoice your clients on a monthly basis. The basic workflow for Autotask integration is as follows:
• Within Autotask, set up a product corresponding to CanIt-Domain-PRO services.
• For each customer, set up a realm within CanIt-Domain-PRO and an account within Autotask.
• For each customer, set up a monthly billing contract within Autotask
• Provide enough information to CanIt-Domain-PRO that it can push usage statistics to Autotask. CanIt-Domain-PRO generates or updates a Contract Cost item, thereby permitting automatic invoice generation.
Once Autotask integration is configured, CanIt-Domain-PRO will automatically post Contact Costs to Autotask with a Unit Quantity corresponding to the number of mailboxes. The Contract Costs will be update every day; this means that whatever your billing cycle is, Autotask will always have up-to-date usage statistics.
5.20.1 Preparing Autotask
To prepare Autotask for CanIt-Domain-PRO integration, perform the following steps within your Autotask account:
Create a Product corresponding to each CanIt-Domain-PRO service
Under Admin : Features & Settings : Products & Services, create a Product for each CanIt-Domain- PRO service that you offer. Once you have finished, the results will look like Figure 5.19:
CanIt-Domain-PRO — AppRiver, LLC 5.20. AUTOTASK® INTEGRATION 89
Figure 5.19: Autotask Product List
The possible products are:
• Inbound Scanning. In this example, we called the product CanIt-Inbound.
• Outbound Scanning. In this example, we called the product CanIt-Outbound.
• Secure Messaging. In this example, we called the product CanIt-SecureMessaging.
• Archiving. You should create one product for each possible retention time in months that you sell. All of these products must have the same prefix, followed by -n where n is the retention time in months. In Figure 5.19, we created three products with the common prefix CanIt-Archiving and retention times of 1, 12 and 24 months.
Create a Recurring Service Contract within Autotask for each CanIt-Domain-PRO cus- tomer
Under Contracts, create a Recurring Service Contract for each CanIt-Domain-PRO customer. Fig- ure 5.20 shows a sample contract, which we have named Email Security:
CanIt-Domain-PRO — AppRiver, LLC 90 CHAPTER 5. CANIT-DOMAIN-PRO SETUP
Figure 5.20: Autotask Recurring Service Contract
5.20.2 Preparing CanIt-Domain-PRO
To prepare CanIt-Domain-PRO for Autotask integration, log in to your top-level realm (the “base” realm if you are running CanIt-Domain-PRO on-premises or your realm if you are using Hosted CanIt. Click on Setup : Autotask Integration. The Autotask settings screen appears:
CanIt-Domain-PRO — AppRiver, LLC 5.20. AUTOTASK® INTEGRATION 91
Figure 5.21: Autotask Integration Settings
Basic Information
Fill in the basic information needed to integrate with the Autotask API. Note that all settings follow realm inheritance; you can override them as necessary on a per-realm basis. The basic settings are:
CanIt-Domain-PRO — AppRiver, LLC 92 CHAPTER 5. CANIT-DOMAIN-PRO SETUP
• Autotask proxy URL: The URL for accessing the Autotask API. The default value is probably fine and should not normally need to be changed.
• Autotask API username: A username with permission to access the Autotask API.
• Autotask API password: The password for the API user.
• Account name associated with current realm: The name of the account within Autotask. This setting links the current realm in CanIt-Domain-PRO to the account in Autotask.
• Contract name associated with email security product: The Autotask contract name corre- sponding to the CanIt-Domain-PRO services.
• Billing metric: One of Addresses or Streams, depending on whether you bill on the basis of number of email addresses or number of streams.
• Bill for subrealms as well as current realm: If set to Yes, then statistics for the current realm and all of its descendants are counted for billing purposes. If set to No, then only statistics within the current realm are pushed to Autotask.
Per-Product Settings
CanIt-Domain-PRO lets you push billing data for up to four product categories to Autotask. The four categories are shown below. Note that you may not offer all categories to all of your clients.
1. Inbound Filtering: Inbound email filtering.
2. Outbound Filtering: Outbound email filtering.
3. Secure Messaging: Secure Messaging Service
4. Archiving: Email archiving.
To link each product to Autotask, fill in the following settings:
• Product name: The name of the corresponding product within Autotask. Note: Archiving is a special case because CanIt-Domain-PRO always appends -n where n is the retention time in months. Therefore, in the Archiving Settings section, the Product name setting specifies the prefix to use.
• Unit cost: The unit cost of the product. For most products, this is simply a decimal number. For Archiving, it is a string of the form: n1=c1,n2=c2,... which specifies that the cost for n1 months of retention is c1, for n2 is c2 and so on. You should enter all combinations of retention time that are actually used by your clients. For example, in Figure 5.21, the Unit cost of 1=0.5,12=1,24=2 means that one month of archiving costs $0.50; 12 months costs $1.00 and 24 months costs $2.00
CanIt-Domain-PRO — AppRiver, LLC 5.20. AUTOTASK® INTEGRATION 93
• Unit cost: The unit price of the product. For most products, this is simply a decimal number. For Archiving, it follows the same format as Unit price.
• Minimum number of units to bill: The minimum number of units to bill each month, if any. For most products, this is an integer, but for Archiving, it follows the same format as Unit price except only integers can appear to the right of each equals sign.
• Contract cost description: If non-blank, the description to use in the contract cost line item. If this is left blank, the description is copied from the Autotask Product.
Once you’ve entered the values for your top-level realm, switch into each customer realm and set (minimally) the Account name associated with the current realm as well as any other settings that should be overridden.
5.20.3 Testing the Autotask Integration Settings
You can test the Autotask settings by enabling the Run a live test of these settings against Autotask checkbox and clicking Submit Changes. CanIt-Domain-PRO will print a debug log and let you know whether or not the settings look correct. Note that because fetching actual statistics is costly, the debugging output always pretends to post random unit counts to Autotask. In production, the correct number of addresses or streams would be posted to Autotask. If you enable the Push a dummy ContractCost item to Autotask checkbox, then in addition to running the tests, CanIt-Domain-PRO will push a ContractCost item up to Autotask A successful debugging log is shown in Figure 5.22.
CanIt-Domain-PRO — AppRiver, LLC 94 CHAPTER 5. CANIT-DOMAIN-PRO SETUP
Figure 5.22: Autotask Test Results
CanIt-Domain-PRO — AppRiver, LLC 5.21. CONNECTWISE® INTEGRATION 95
Sample Contract Costs as they appear in Autotask after CanIt has pushed statistics are shown in Figure 5.23.
Figure 5.23: Autotask Contract Costs
5.20.4 Autotask Settings and Inheritance
In the Autotask Integration Settings screen, the “Origin Realm” column shows the realm in which a setting has been created. If the setting exists in the current realm, you can check the Reset? checkbox to remove the setting and make the setting once again inherit the value from the parent realm. Any realm that does not have an Autotask account name associated with it will not have its statistics pushed to Autotask.
Note: The first time CanIt-Domain-PRO connects to Autotask, it extracts the Account, Contract and Product IDs from Autotask and from then on uses the Autotask IDs rather than the names to link to Autotask. This allows you to rename objects within Autotask without breaking the CanIt-Domain-PRO integra- tion. As a convenience, if CanIt-Domain-PRO notices that an object has been renamed in Autotask, it updates its copy of the corresponding name to match Autotask’s.
5.21 ConnectWise® Integration
CanIt-Domain-PRO can automatically update mailbox counts in a ConnectWise Agreement Addi- tion. These updates are done nightly, meaning that whenever your billing cycle falls, the Agreement Addition will have up-to-date counts.
5.21.1 Preparing ConnectWise
1. Create a new Product for each CanIt service you offer. The product names must be as follows; you only need to create those products that you are actually using. • CanIt-Inbound for inbound email filtering. • CanIt-Outbound for outbound email filtering. • CanIt-SecureMessaging for secure messaging. • CanIt-Archiver-n for archiving with a retention time of n months. You need to create one CanIt-Archiver-n Product for each retention time you offer.
CanIt-Domain-PRO — AppRiver, LLC 96 CHAPTER 5. CANIT-DOMAIN-PRO SETUP
See Figures 5.24 and 5.25 for examples of how to create the Products within ConnectWise.
Figure 5.24: CanIt-Inbound ConnectWise Product
CanIt-Domain-PRO — AppRiver, LLC 5.21. CONNECTWISE® INTEGRATION 97
Figure 5.25: CanIt Product List
CanIt-Domain-PRO — AppRiver, LLC 98 CHAPTER 5. CANIT-DOMAIN-PRO SETUP
2. Set up an Integrator Login ID and Password Under System : Setup Tables : Integrator Login, create a login for CanIt-Domain-PRO to ac- cess the API. The login that you create must be able to access the following APIs: Managed Services API, Company API, Product API, Reporting API, System API and Agreement API. See Figure 5.26.
Figure 5.26: Integrator Login ID Setup
CanIt-Domain-PRO — AppRiver, LLC 5.21. CONNECTWISE® INTEGRATION 99
3. Create a Management IT Solution for CanIt-Domain-PRO billing. Under System : Setup Ta- bles : Management IT Solution List, create a Management IT Solution. The name must be CanItBilling and the Management IT Solution should be Custom. See Figure 5.27.
Figure 5.27: CanItBilling Management IT Solution Setup
CanIt-Domain-PRO — AppRiver, LLC 100 CHAPTER 5. CANIT-DOMAIN-PRO SETUP
4. Set up Managed Devices Integration. Under System : Setup Tables : Managed Devices In- tegration List, add a CanItBilling entry with the solution set to CanItBilling. The Integrator Login should be set to the login name you made in Step2 earlier. See Figure 5.28.
Figure 5.28: CanItBilling Managed Device Integration Setup
CanIt-Domain-PRO — AppRiver, LLC 5.21. CONNECTWISE® INTEGRATION 101
5. Create an Agreement for your customer, if there isn’t one yet. See Figure 5.29.
.
Figure 5.29: Connectwise Agreement
CanIt-Domain-PRO — AppRiver, LLC 102 CHAPTER 5. CANIT-DOMAIN-PRO SETUP
6. Create a new Agreement Addition (if one does not yet exist) for each CanIt product that you will bill for. See Figure 5.30.
.
Figure 5.30: Connectwise Agreement Addition
CanIt-Domain-PRO — AppRiver, LLC 5.21. CONNECTWISE® INTEGRATION 103
5.21.2 Preparing CanIt-Domain-PRO
1. In your main realm, click on Setup and then ConnectWise® Integration. The ConnectWise setup page appears (Figure 5.31):
Figure 5.31: ConnectWise Setup - Main Realm
In the main realm, all you should enter are:
• ConnectWise Web Site URL: the URL for accessing the ConnectWise API. If you are unsure what it is, please contact your ConnectWise adminis- trator or ConnectWise technical support. If your regular URL is some- thing like na.myconnectwise.net, then the API URL is probably api-na.myconnectwise.net/v4 6 release/apis/2.0. • ConnectWise Company for Login: The company name you use to log into ConnectWise. • ConnectWise Username for Login: The username you created in Step2 in Section 5.21.1. • ConnectWise Password: The password you created in Step2 in Section 5.21.1. • Agreement Name: You can fill in a default Agreement Name to use for all of your billing purposes. • Billing Metric: Choose Addresses if you are billing based on the number of email ad- dresses, or Streams if based on streams. • Bill for Subrealms as Well: Set to Yes if you want to bill a realm for its own mailboxes and those of its subrealms, or No if you only want to bill for mailboxes within the specific realm.
CanIt-Domain-PRO — AppRiver, LLC 104 CHAPTER 5. CANIT-DOMAIN-PRO SETUP
• Bill Customer: This setting allows you to override the BillCustomer flag; select one of Billable (the default), DoNotBill or NoCharge. • Unit Price: You may optionally override the unit price by entering a decimal number. • Unit Cost: You may optionally override the unit cost by entering a decimal number.
You should not fill in anything for the Company Name Associated with this Realm since this is specific to each customer being billed.
2. For each customer realm that should be billed, switch into that realm and click Setup and then ConnectWise® Integration. In this page, enter the Company Name associated with the realm; it must exactly match the Company name in ConnectWise. You can also override other settings such as Agreement Name, Billing Metric, Bill for Subrealms as Well, Bill Customer, Unit Price and Unit Cost, if necessary. Also, make sure the Agreement Name matches the Agreement you set up in Step5 in Section 5.21.1. You should test the connectwise settings by enabling “Run a live test of these settings against ConnectWise” and “Update AgreementAddition data on ConnectWise”. Then click Submit Changes. If all goes well, the test results will look something like Figure 5.32.
Figure 5.32: ConnectWise Test Results
CanIt-Domain-PRO — AppRiver, LLC Chapter 6
CanIt-Domain-PRO Administration
6.1 Global Settings
Note: This section describes features that only the CanIt-Domain-PRO System Administrator can use. The first administrative task you should undertake is to set up global settings. Click on the Adminis- tration link. You will see the global settings screen:
Figure 6.1: Global Settings
Note that the Basic Setup Wizard (Section 5.3.1) populates some of these settings. The “ID” column is a unique identifier for each setting; it is not used except as a convenient way for AppRiver support personnel to indicate a particular setting over the phone. The global settings have the following meanings:
G-1100 Maximum size of message to scan for spam (kB) Spam-scanning can be very slow on large
CanIt-Domain-PRO — AppRiver, LLC 105 106 CHAPTER 6. CANIT-DOMAIN-PRO ADMINISTRATION
messages. If a message comes in that is larger than this threshold, CanIt-Domain-PRO attempts to reduce its size by removing non-text attachments before feeding the message to the scanning engine. If this succeeds, the reduced message is scanned. If the message is still too large even after the reduction, it is not scanned for spam.
G-2400 Handling for messages containing viruses If you have a virus-scanner compatible with CanIt-Domain-PRO, this setting controls how CanIt-Domain-PRO deals with virus-bearing messages. Hold holds the message in the quarantine for approval (or tags the message if the stream is in tag-only mode.) Accept permits the message to pass, while Reject rejects it with an SMTP failure code. Finally, Discard simply discards the message. We recommend setting this option to Discard. Note: This setting may be overridden on a per-stream basis.
G-1500 Expire statistics after this many days Once a day, a cron job removes old entries from the statistics table. By default, CanIt-Domain-PRO keeps statistics for 10,000 days (around 27 years), but you can lower this setting to as low as 90 days if you do not want to keep old statistics around.
G-1550 Number of hours to keep detailed statistics CanIt-Domain-PRO keeps very detailed statis- tics for a limited time. This setting lets you adjust the length of this time.
G-1600 Expire old data after this many days Once a day, a cron job purges old messages, log entries and incidents from the database. We recommend retaining at least 14 days’ worth of data, although you might want to lower this on a busy mail server. Note: This setting is the number of days from the creation of the incidents being expired, regardless of whether or when they were marked as spam or non-spam.
G-1610 Remember change history for this many days Most CanIt-Domain-PRO web pages have a “Show Changes” link that lets you see changes made to rules and settings. This setting specifies how long change history should be retained. It may be set to any integer from 45 to 10000 and defaults to 732 days (about two years).
G-1700 Expire messages marked as spam after this many days This setting controls when the cron job expires messages you have marked as spam. Note that it only applies to closed incidents— that is, messages that have not only been marked as spam, but have also actually been rejected by CanIt-Domain-PRO.
G-1800 Expire messages marked as non-spam after this many days This setting controls when the cron job expires messages you have marked as non-spam. Note that it only applies to closed incidents—that is, messages that have not only been marked as non-spam, but have also actually been delivered by CanIt-Domain-PRO.
G-4010 Number of hours to cache address-to-stream lookups As mentioned in Section 2.4, address-to-stream mappings may be cached in the Address Mapping Table. This setting specifies for how long cached entries remain valid.
G-4015 Number of hours before refreshing cached address-to-stream lookups If a cached address is older than this many hours, CanIt-Domain-PRO attempts to perform an address-to-stream
CanIt-Domain-PRO — AppRiver, LLC 6.1. GLOBAL SETTINGS 107
mapping to refresh the cached entry. If the lookup fails with a temporary failure, CanIt-Domain- PRO does not update the cached entry, but will continue to use it until it expires as per setting G-4010. If the lookup succeeds, CanIt-Domain-PRO updates the cached entry. If it fails with a “No such user” result, CanIt-Domain-PRO deletes the cached entry.
G-4050 Time in hours to delay messages with Delayed Attachments If you use the Delayed At- tachments feature, this setting controls the length of the delay.
G-4800 Number of days to keep mail signatures for Bayesian analysis This setting specifies how long after a message first arrives a user may vote on whether it is spam or non-spam.
G-4900 Number of generations before cleaning common Bayes tokens CanIt-Domain-PRO pe- riodically cleans old data out of the Bayes database. This setting controls how long CanIt-Domain-PRO retains a token that has been seen frequently, but not recently. We recommend leaving it at the default value.
G-5000 Number of generations before cleaning uncommon Bayes tokens CanIt-Domain-PRO pe- riodically cleans old data out of the Bayes database. This setting controls how long CanIt- Domain-PRO retains a token that has been seen infrequently and not recently. We recommend leaving it at the default value.
G-4020 Users must opt in to anti-spam scanning? If you set this to Yes, then users must explicitly opt-in to anti-spam scanning. If users do not opt-in, their mail is simply passed through un- changed. If you set this to No, then all users are implicitly opted-in. They can, however, explicitly opt out if they choose.
G-4030 Users must be approved for anti-spam scanning? If you set this to Yes, then the CanIt- Domain-PRO administrator’s approval is required before a user can opt in to anti-spam scan- ning. If you are selling anti-spam scanning as a value-added service, you should set this to Yes. If anti-spam scanning is part of your basic service, set it to No. Note that opting in and opting out is done on a per-stream basis. Usually, a stream corresponds to a user, but it is possible for a stream to correspond to more than one user, and for a single user to be responsible for more than one stream.
G-4300 Minimum size of spam corpus for Bayesian analysis CanIt-Domain-PRO will not use Bayes data until at least this many messages have been trained as spam.
G-4400 Minimum size of non-spam corpus for Bayesian analysis CanIt-Domain-PRO will not use Bayes data until at least this many messages have been trained as non-spam.
G-3600 Always allow users who use SMTP authentication If your version of Sendmail is compiled to support the SMTP AUTH extension, you can always allow mail from authenticated senders by setting this to Yes. (The default is No.) In this case, mail from authenticated users will not be scanned for spam (but will still be scanned for viruses and bad filename extensions or MIME parts.)
Note: CanIt currently cannot preserve SMTP AUTH-based allow-rules when messages are streamed. Thus, if an AUTH’ed user sends mail to recipients in more than one stream, the allow rule will not be applied.
CanIt-Domain-PRO — AppRiver, LLC 108 CHAPTER 6. CANIT-DOMAIN-PRO ADMINISTRATION
G-3900 Store both raw and decoded messages in incident database Some e-mail messages are ob- scured using Base64 encoding or some other encoding scheme. If you change this setting to Yes, CanIt-Domain-PRO stores both the “raw” and “decoded” message in the incident database. This lets you view encoded messages more reliably, but approximately doubles the disk space used by the incident database. If you set it to No (the default), CanIt-Domain-PRO stores only the raw message. The message display Web page can decode some encoded messages, but it is not completely reliable. If you need a completely reliable way to view encoded messages, you should change this setting to Yes.
G-4000 Obscure To, Cc and Bcc fields for non-root users Because CanIt-Domain-PRO stores mes- sages that hash identically only once, the To:, Cc: and Bcc: headers of messages may leak recipient information to other recipients of the message. To hide this information, change this setting to Yes.
G-4060 Users authenticated by external means default to simple GUI? If you set this to Yes, then users who authenticate via an external authentication mechanism have a much simplified inter- face to CanIt-Domain-PRO by default. This simplified interface is described in Chapter 10.
G-4075 Switching to expert mode cancels stream inheritance If you use the Simple Interface (Chapter 10), then you may wish to cancel inheritance whenever a user selects the expert in- terface. In that case, change this setting to Yes. That is, if a user has selected a particular spam-scanning level in the Simple Interface, then when they switch to Expert Interface, the se- lected level is no longer used—instead, individual settings are used that do not depend on any of the preconfigured spam-scanning settings.
G-4080 Support the Sendmail ‘plus hack’ for streaming Some Sendmail configuration files allow users to add a “+” sign followed by arbitrary text to their user names, and use the resulting e-mail addresses for various purposes such as filtering e-mail. If you change this setting to Yes, then CanIt-Domain-PRO ignores a “+” sign and any following text after the user name part when mapping e-mail addresses to streams. Note that if you use the “Program” method to stream e-mail, the “+” sign and any following text is retained; it is up to your program to implement the sendmail “plus hack” if you choose.
G-4090 Scan for viruses prior to streaming incoming mail If you know for sure that you always want to reject or discard viruses, regardless of any per-stream settings, then change this setting to Yes. It causes any viruses to be discarded or rejected (according to the global virus-handling setting) before any streaming takes place. If a virus comes in for more than one recipient, this can greatly reduce the load on CanIt-Domain-PRO. Note that the global virus-handling setting must not be set to Hold/Tag for this setting to take effect.
G-4100 Timeout in seconds for Verification Server queries If you are using the Verification Server feature, CanIt-Domain-PRO will time out Verification Queries according to the value of this setting. You should keep it reasonably low so that a slow or dead verification server does not interfere with delivery to other domains.
To make your changes permanent:
CanIt-Domain-PRO — AppRiver, LLC 6.2. SRS (SENDER REWRITING SCHEME) 109
• Click on Update Global Settings
6.2 SRS (Sender Rewriting Scheme)
In order to avoid spurious SPF failures when CanIt-Domain-PRO forwards mail to a back-end server that performs SPF checking, you can enable Sender Rewriting Scheme (see http://en. wikipedia.org/wiki/Sender_Rewriting_Scheme for a description of Sender Rewriting Scheme.) To enable Sender Rewriting Scheme, you must perform the following steps:
• Pick a domain to use for the SRS addresses. This domain should not be currently in use for anything else. We recommend creating a subdomain of your existing domain solely for use with SRS. For example, if you own the domain example.com, then srs.example.com would be a good choice.
• Publish MX records for the SRS domain that point to the CanIt-Domain-PRO scanner or scan- ners.
• Under Administration, enter the SRS domain as the value of G-11000 SRS Domain
• If and only if you are not running a CanIt-Domain-PRO appliance, perform the following steps:
1. Update the Sendmail access map to permit relaying for the SRS domain. 2. Add a mailertable entry for the SRS domain and set the mailer to local:srshandler 3. Create a Sendmail alias directing srshandler to "|/usr/share/canit/scripts/canit-srs-bounce-handler"
If you are running a CanIt-Domain-PRO appliance, the above steps are done for you automati- cally.
Additionally, you must specifically enable SRS on a per-stream basis (following the usual CanIt- Domain-PRO inheritance rules.) To turn on SRS for a stream, enable setting S-930 “Enable SRS (Sender Rewriting Scheme)” under Preferences : Quarantine Settings.
Note: SRS requires Sendmail 8.14 or newer. If you are not running a CanIt-Domain-PRO appliance, make sure you have a new enough version of Sendmail. Once SRS is enabled, CanIt-Domain-PRO will rewrite envelope senders that receive SPF “pass” to addresses within the SRS domain. CanIt-Domain-PRO will also handle bounces to those addresses, restoring the original recipient. Here are a few items to note about SRS:
• CanIt-Domain-PRO does not apply SRS to mail that was forced into a stream by a Known Networks entry. Such mail is typically outbound mail; in this case you should simply include the outbound CanIt-Domain-PRO relays’ IP addresses in the domain’s SPF record.
CanIt-Domain-PRO — AppRiver, LLC 110 CHAPTER 6. CANIT-DOMAIN-PRO ADMINISTRATION
• If you have back-end servers that forward inbound mail back out via CanIt-Domain-PRO (this can happen, for example, if some users on the back-end server configure their accounts to for- ward everything to Gmail or to Hotmail) then you should enable SRS on the inbound mail for those users.
• CanIt-Domain-PRO applies SRS only if the original inbound mail received and SPF “pass”.
6.3 Real-Time DNS Blocklists
Note: This section describes features that only the CanIt-Domain-PRO System Administrator can use. Both Sendmail and CanIt-Domain-PRO can make use of DNS-based real-time blocklists. These blocklists allow you to look up the IP address of a host in a special DNS domain, and take action if the host is on the list. You can configure Sendmail to use DNS-based blocklists directly, but you may prefer to handle this with CanIt-Domain-PRO, because CanIt-Domain-PRO allows you to hold or score messages from hosts on the blocklist rather than outright rejecting them.
6.3.1 Entering the Master List of DNS RBLs
To use DNS-based RBLs, you first enter a master list of RBLs that CanIt-Domain-PRO can potentially use. To do this, click on Administration and then Master RBLs. The Master RBLs page appears:
Figure 6.2: Master RBLs
To enter an RBL:
1. Enter the domain in the RBL Domain box.
2. Enter a brief (but meaningful) description in the Description box.
3. Enter a short tag in the Tag box. This tag is used in the mail log and incident reports to identify the RBL. If you leave it blank, CanIt-Domain-PRO will construct a unique identifier for the RBL based on the domain, type and data.
4. Select how the RBL is to be used:
CanIt-Domain-PRO — AppRiver, LLC 6.3. REAL-TIME DNS BLOCKLISTS 111
(a)A Block RBL is used to block unwanted mail. Users will be able to create “Ignore”, “Hold/Tag”, “Reject” or “Score” RBL rules. Any “Score” rule will have to have a non- negative score. (b) An Allow RBL is used to list known good mail servers. Users will be able to create “Ignore” or “Score” rules, but any “Score” rule will have to have a non-positive score. In addition, no extra greylist delay may be created for an Allow RBL. 5. Select the type of addresses listed by the RBL: (a) If you know that the RBL lists only IPv4 addresses, set the Address Family to IPv4. (b) If you know that the RBL lists only IPv6 addresses, set the Address Family to IPv6. (c) If the RBL lists both IPv4 and IPv6 addresses, set the Address Family to Both IPv4 and IPv6. If you are not certain whether or not the RBL lists IPv6 addresses, the “Both” setting is safest. 6. Select the type of the RBL: (a) If the RBL is considered to be “hit” if any record is returned, set the type to normal. Most DNS-based blocklists are of this type. (b) If the RBL returns specific A records to indicate a hit, set the type to match and enter the A record that indicates a hit in the Data field. As a special case, you can use an X in place of an octet to allow a wildcard match. For example, a data field of 127.0.X.3 would match an A record of 127.0.0.3, 127.0.1.3, 127.0.55.3, etc. (c) If the RBL returns information in a bitmask in the returned A record, set the type to mask and enter the mask (for example, 0.0.0.4) in the Data field. A mask-type RBL is considered to be hit if the returned A record bitwise-ANDed with the data field returns non-zero. 7. Click Submit Changes
To delete an RBL, enable the checkbox beside the entry you wish to delete and click Submit Changes. Deleting a master RBL also deletes all RBL rules that refer to it. You can change the timeout for RBL lookups by adjusting the value in the Timeout in seconds for DNS-RBL lookups box. The master RBL list is merely a list of all the RBLs that CanIt-Domain-PRO can potentially use. To actually set up RBL rules, please see the User’s Guide. RBL rules can be created on a per-stream basis, so different streams can elect to use none, some or all of the predefined Master RBLs.
Note: Various RBLs have different terms-of-service. Some require licensing or payment; please be sure you are allowed to use an RBL before entering it into CanIt-Domain-PRO’s RBL list.
6.3.2 combined.bl.rptn.ca
AppRiver LLC publishes for DNS-based lists for CanIt-Domain-PRO customers. These lists are au- tomatically entered into the Master RBL list (but no rules are created automatically.) The four lists are:
CanIt-Domain-PRO — AppRiver, LLC 112 CHAPTER 6. CANIT-DOMAIN-PRO ADMINISTRATION
• The Greylist-Stumbler list. These are machines known to have trouble getting past greylisting. The machines are very likely compromised PCs. We recommend making a rule to add 0.5 points for machines on this list, and also to extend the greylist period (if you use greylisting) to 60 minutes.
• The Dictionary-Attacker list. These are machines known to send mail to many nonexistent addresses. We recommend a rule to add 0.5 points for machines on this list.
• The Spam-Source list. These are machines known to send spam and relatively little non-spam. We recommend adding three points for machines on this list.
• The Mixed list. These machines send both spam and non-spam; we recommend adding 1.5 points for these machines.
Note: The combined.bl.rptn.ca list requires a secret token for lookups to succeed; this token is changed once a day. CanIt-Domain-PRO automatically obtains and uses the token for as long as your support term is in force. This means that you cannot use the list outside of CanIt-Domain-PRO. If you do a high volume of lookups, please contact AppRiver, LLC to arrange for a zone transfer via rsync.
6.4 Phishing URLs
Note: The ability for end-users to vote URLs as malicious is available only if you have enabled CanIt Storage Manager (Chapter 16) CanIt-Domain-PRO maintains a list of URLs that are known to be malicious or to have been used in phishing messages. There are two sources of these URLs:
• A large list is distributed by AppRiver to each CanIt-Domain-PRO installation. Your RPTN credentials provide access to this list.
• Each CanIt-Domain-PRO administrator can additionally maintain a local list of phishing URLs.
6.4.1 Malicious URL Votes
When end-users reject an incident from the quarantine page, they can choose merely Reject message or the stronger Reject and Report Phish/Fraud. The latter presents users with a list of URLs in the rejected message and asks them to indicate which URLs they believe to be malicious. Each such URL is entered as a phishing URL vote. You can review phishing URL votes by clicking on Administration and then Phishing URLs. The Phishing URL Votes page appears:
CanIt-Domain-PRO — AppRiver, LLC 6.4. PHISHING URLS 113
Figure 6.3: Phishing URL Votes
This page shows all of the URLs that users have indicated are malicious. The various columns in the display are:
• URL — a normalized version of the URL with any leading http: or https: stripped. Note that URLs longer than 40 characters are truncated and an ellipsis (...) is placed after them; hover the mouse pointer over the URL to see the full URL. • Votes — the number of times the URL has been voted as malicious. • In Phishing List? — Set to “No” if the URL is not in the central list of known phishing URLs, or “Bad” if it is. If the URL has query parameters (for example: example.com/foo?x=1) and the base URL example.com/foo without query parameters is in the central list, then this column will contain “Base URL Bad”. If the URL is in the known phishing list, then the source is indicated as local or RPTN:*. local means the URL was added by the local CanIt-Domain-PRO administrator; RPTN:* means it came from the central AppRiver list. The specific text after RPTN: provides additional detail about the source of the URL. • Last Vote — the date the URL was last voted as being malicious. • Action — a list of actions to take against the URL. Possibilities are: – Do Nothing — don’t take any action. – List URL as Bad — enter the URL into the known phishing URL list, marked as mali- cious. – List base URL as Bad — remove the query parameters, if any, from the URL and enter it as a malicious URL in the known phishing URL list. – List URL as OK — explicitly indicate that the URL is not malicious. You can use this if, for some reason, you need to override a URL marked as malicious in the central AppRiver list. Note that any URL you enter into the list of known phishing URLs will be set to expire after 120 days. You can alter this expiry date as described in Section 6.4.2. • Delete? — this permits you to delete all votes relating to the URL. Note that if the URL is in the known phishing URL list, deleting it from the Phishing URL Votes page does not remove it from the list. It merely deletes all users’ votes pertaining to the URL.
CanIt-Domain-PRO — AppRiver, LLC 114 CHAPTER 6. CANIT-DOMAIN-PRO ADMINISTRATION
If you take action against phishing URLs or delete any, click Submit Changes to make your changes take effect.
Filtering the List of Phishing URL Votes
You can filter the list of phishing URL votes displayed as follows:
• Enter a string in the “Entry Contains:” filter box to restrict URLs to those containing a particular string.
• Enter a positive integer in the “Minimum Votes:” filter box to restrict URLs to those with at least that many votes.
• Select one of “Any”, “Yes” or “No” from the “In Known-Phishing List?” pulldown to restrict the URL display to those which meet the filter condition.
Once you have created filter conditions, click Filter to apply them.
6.4.2 Known Phishing URLs
To see the entire list of URLs known or suspected to be malicious, click on Administration and then Phishing URLs. In the Phishing URL Votes page, click on Known-Phishing List in the third-level menu. The Known Phishing URLs page appears:
Figure 6.4: Known Phishing URLs
The list of phishing URLs has eight columns:
• URL — a normalized form of the URL. Note that URLs longer than 40 characters are truncated and an ellipsis (...) is placed after them; hover the mouse pointer over the URL to see the full URL.
• Votes — the number of times a URL has been voted as malicious by a local user.
• Status — “Bad” if the URL is considered malicious; “Good” if it is considered harmless.
• Source — the source that determined the URL to be malicious. Possible values for Source are:
CanIt-Domain-PRO — AppRiver, LLC 6.4. PHISHING URLS 115
– local — the URL was marked as malicious by the local CanIt-Domain-PRO site adminis- trator. – RPTN:APER — the URL was considered malicious by the Anti-Phishing Email Reply project at https://code.google.com/p/anti-phishing-email-reply/. – RPTN:Phishtank — the URL was considered malicious by the Phishtank project at http://www.phishtank.com/.
If AppRiver adds additional feeds of malicious URLs, there may be additional values for Source, but all of them will start with RPTN:.
• Last Vote — the time of the most recent vote by a local user (if there was one) that the URL was malicious.
• Expiry — the date when the URL will expire and be auto-deleted from the list. By default, local entries expire 120 days after they are created. RPTN entries do not expire, but are removed if the URL is removed from the central RPTN lists maintained by AppRiver.
• Action — an action to take against the URL. Possible actions are Do Nothing, List URL as Bad and List URL as OK, all of which are self-explanatory.
• Delete? — a checkbox for deleting a URL from the known phishing URL list. Note: If you delete a URL with a source other than local, it will reappear next time CanIt-Domain-PRO updates its URL list from AppRiver’s data feed.
If you make any changes (taking action against URLs, changing the expiry date or deleting any URLs), click Submit Changes to make them take effect. If you wish to add URL that is not currently in the Known Phishing URLs list, you can enter it in the top row in the URL column and hit Submit Changes to add it to the list manually.
Filtering the Known Phishing URL List
You can restrict which URLs are displayed by entering text into the “Entry contains:” and/or “Source Contains:” filter boxes and clicking Filter.
6.4.3 Delaying Messages because of local Phishing Votes
There can be a significant delay between the time a URL is voted on by end-users as fraudulent and the time the administrator adds it to the Known Phishing list. To mitigate problems caused by this delay, CanIt-Domain-PRO allows you (on a per-stream basis) to delay messages once the URLs in them have a certain number of phish votes. Under Preferences : Quarantine Settings, set S-1630 to the minimum number of votes to trigger a delay. A value of 0 disables the feature. We recommend a value of at least 5 so that messages are unlikely to be delayed because of a couple of incorrect votes. On that same page, set S-1640 to the number of hours to delay the message.
CanIt-Domain-PRO — AppRiver, LLC 116 CHAPTER 6. CANIT-DOMAIN-PRO ADMINISTRATION
If a message comes in containing a URL that has at least as many phish votes as the S-1630 setting, then it is put into a special stream called @@DELAYED in the recipient’s realm. After the number of hours specified in S-1640, the message will automatically be released from @@DELAYED and res- canned. We recommend setting a notification address in the @@DELAYED stream to notify administrators hourly. That way, they can check that stream’s quarantine and reject malicious messages before they are released. Administrators can also take the opportunity to add malicious voted-on URLs to the Known Phishing URL list.
6.5 Users
CanIt-Domain-PRO maintains its own table of users. You should enter users into this table to create CanIt-Domain-PRO administrative users, or users with different privileges from the default (for ex- ample, a demo user.) Click on Administration and then Users to set up users. You will see the user management screen.
Figure 6.5: Users
If you enter a string in the “Filter:” box, then CanIt-Domain-PRO limits the display to entries whose User-ID or E-Mail column contain that string. We recommend using the form [email protected] for user names. Users whose names follow that format will be placed in the appropriate realm, as determined by looking up domain.com in the Realm Mapping Table. A user name of the form user without a domain will normally be placed in the base realm. (However, a user can specify a particular realm to log in to by logging in as realm:user.)
CanIt-Domain-PRO — AppRiver, LLC 6.5. USERS 117
6.5.1 User Privileges
When a user logs in to CanIt-Domain-PRO, he or she can see a single stream at a time. Every user always has access to a stream that (usually) has the same name as his user name. The CanIt- Domain-PRO administrator can give users permission to see additional streams. For example, the user janedoe always has access to the stream janedoe. However, if she manages a mailing list called joke-list, you have two options:
1. You can stream messages for the list to janedoe, so she has only a single spam quarantine to consider.
2. You can create a new stream called joke-list and give access to that stream to janedoe. In this way, she can use different settings, blocklists and allow-lists for the list than she does for her personal e-mail.
Each CanIt-Domain-PRO user has two special privileges, which can be on or off:
• A user with root privilege can add, edit and delete other users. A user with root privilege in the base realm has overall System Administrator privileges. A user with root privilege in any other realm has Realm Administrator privileges. The overall System Administrator can see (and create) users in other realms. Realm Administrators can only create users in their own realms.
• A user with write privilege can mark messages as spam or not-spam, and can block and allow hosts, domains and senders. A user without write privilege is called a read-only user and cannot make any changes whatsoever. A read-only user can look, but not touch.
Note that CanIt-Domain-PRO allows for additional flexibility in controlling which parts of the Web interface are available to various users. For details, see Chapter9.
6.5.2 Adding a User
To add a user, click on the Add User link. The Add User screen appears:
CanIt-Domain-PRO — AppRiver, LLC 118 CHAPTER 6. CANIT-DOMAIN-PRO ADMINISTRATION
Figure 6.6: Add User
• Enter the user-ID of the user in the User-ID box.
• Select the realm for the new user from the Realm pull-down menu or enter it into the realm box. If you leave the realm blank, then a realm will automatically be assigned based on the user-ID. If the user-ID looks like an e-mail address, the realm is chosen by mapping the domain-name part of the user-ID to a realm.
• To set the user’s e-mail address, enter it in the E-Mail field. (If CanIt-Domain-PRO knows a user’s e-mail address, the “Locked Addresses” feature can be used.)
• Enter a password for the user in the Password and Confirm Password fields.
• If you set Locked Password? to Yes, then the user will have a “locked” password and will not be able to log in. However, if you have configured an alternate user authentication method, the user will be able to log in using a password that the alternate method accepts.
• If you only want the user to have read-access to the spam quarantine, set Write Access? to No.
• If you want to make the user an administrator in his realm, set Has Root Access? to Yes.
Once you have filled in the fields, click Add User to add the user.
Note: Both user-names and passwords are case-sensitive; a used named user1 is completely different from one named User1.
6.5.3 Editing a User
To edit a user, click on the User-ID on the user management screen. You will see the user-editing screen.
CanIt-Domain-PRO — AppRiver, LLC 6.5. USERS 119
Figure 6.7: Edit User
• The user’s realm may be displayed, but it cannot be edited once the user has been created.
• To set the user’s e-mail address, enter it in the E-Mail field.
• If you wish to change the user’s password, enter it in the Password and Confirm Password fields. If you leave these fields blank, the password will not be changed.
• If you set Locked Password? to Yes, then the user will have a “locked” password and will not be able to log in. However, if you have configured an alternate user authentication method, the user will be able to log in using a password that the alternate method accepts.
• Adjust the write-access privilege by setting the Write-Access? checkbox appropriately. (If you are editing the currently logged-in user, you can’t change the Write-Access setting.)
To make the changes take effect, click Submit Changes.
6.5.4 Deleting a User
If there is more than one user, a Delete checkbox appears beside those users that can be deleted. Enable the checkbox and then click Submit Changes to delete the selected user or users. Note that it is not possible to undo the deletion! Note that if you delete a user, he may still have access if he can be authenticated using an external authentication mechanism.
6.5.5 Granting Access to Streams
If you wish to grant a user access to additional streams, click on the Edit Accessible Streams button (Figure 6.7). The following page will appear:
CanIt-Domain-PRO — AppRiver, LLC 120 CHAPTER 6. CANIT-DOMAIN-PRO ADMINISTRATION
Figure 6.8: Granting Access to Streams
To grant access to a stream, enter the stream name in the input box and click Add Stream. To revoke access to a stream, enable the Delete checkbox next to the stream name and click Delete Selected Streams. If you grant access to a stream named * (a single asterisk), then the user is given access to all streams in his or her realm. Note that a user always has access to a stream with the same name as his user name, and this access cannot be revoked. Also, the CanIt-Domain-PRO administrator can access any stream, regardless of the settings on this page.
6.5.6 Switching Users
CanIt-Domain-PRO permits an administrative user to switch to another user-ID. This is useful if you want to see the interface exactly as another user would see it. A realm administrator can switch to any user within his own realm or any realm in the subtree under that realm. To switch users:
1. Click Administration : Switch User
2. Enter the user you wish to become in the User-ID box.
3. Enter the stream in which the user should be placed after the switch in the Stream box. Note that CanIt-Domain-PRO does not run the normal user-lookup to determine a user’s home stream when you switch users; hence, you may need to enter the home stream explicitly.
4. If you own subrealms, you will be asked for the realm of the new user. Note that CanIt-Domain- PRO does not attempt to deduce the realm based on the User-ID; you need to explicitly select a realm in the Realm field.
5. Click Submit Changes. You are now logged in as the new user.
Note: Once you switch users, there is no going back. In most cases, you have to log out and log back in again to become the original user. Also, if you are logged in as a read-only user, then you remain read-only no matter which user you switch to.
CanIt-Domain-PRO — AppRiver, LLC 6.6. PERMITTING USERS TO OPT IN 121
6.6 Permitting Users to Opt In
In the CanIt-Domain-PRO global settings (Section 6.1), the CanIt-Domain-PRO administrator can control:
• Whether or not people are permitted to opt-in to spam scanning. • Whether the default setting is opt-in or opt-out.
There are three useful combinations:
1. Permit everyone to opt-in, and have the default be opt-in. 2. Permit everyone to opt-in, and have the default be opt-out. 3. Permit only selected people to opt-in, and have the default be opt-out.
In the first two cases, the administrator need not do anything special. In the third case, you must add entries to the Stream Approval Table. Click on Administration and then Opt Others In/Out to see this table:
Figure 6.9: Stream Opt-In Approval
If the “Approved?” column is checked, then the stream may opt in to spam scanning. If it is not checked, then the stream may not opt in to spam scanning. If the “Opted-In?” column is checked, the stream is currently opted in to spam scanning. Otherwise, it is not. To add a stream to the table, enter the stream name in the input box and set “Approved?” and “Opted- In?” appropriately. Then click Submit Changes. To edit existing streams, adjust “Approved?” and “Opted-In?” appropriately and click Submit Changes. To delete a stream from the opt-in table, enable the Delete? checkbox on the appropri- ate row and click Submit Changes.
CanIt-Domain-PRO — AppRiver, LLC 122 CHAPTER 6. CANIT-DOMAIN-PRO ADMINISTRATION
If the default setting is to permit anyone to opt in to spam scanning, you can nevertheless exclude particular streams from being able to opt in by entering them in the Stream Approval Table and turning off the “Approved?” checkbox. In order for spam-scanning to occur, a stream must be both approved and opted-in. If the stream is not found in the Stream Approval Table, then the defaults are taken from the Global Settings. If you enter a string in the “Filter:” box, then CanIt-Domain-PRO limits the display to entries whose Stream column contains that string.
6.7 Groups
For the purpose of granting permissions, CanIt-Domain-PRO allows you to create groups. A group is simply a collection of users. To edit groups, click on Administration and then Groups. The Groups Page appears:
Figure 6.10: Groups
6.7.1 Creating, Deleting and Editing Groups
To create a new group:
1. Enter the name of the group in the Group box.
2. Enter a description of the group in the Description box.
3. Click Submit Changes
To delete an existing group:
1. Enable the Delete checkbox for the group you want to delete.
2. Click Submit Changes
To edit a group:
CanIt-Domain-PRO — AppRiver, LLC 6.8. VIEWING ACTIVE STREAMS 123
1. Click on the Edit link next to the appropriate group. The Group Members page appears:
Figure 6.11: Group Members
2. Enter new members (one per line) in the Member text area.
3. If you want to delete existing members, enable the appropriate Delete checkbox.
4. Click Submit Changes
Note: External authentication methods can affect group membership. See Chapter7 for details. In the Groups Page (Figure 6.10), click on Permissions to edit the permissions associated with the group. Permissions will be discussed in detail in Chapter9.
6.8 Viewing Active Streams
The CanIt-Domain-PRO administrator can look at all the streams with entries in the incidents table. To do this, select Administration and then See Active Streams. The Active Streams Page appears:
CanIt-Domain-PRO — AppRiver, LLC 124 CHAPTER 6. CANIT-DOMAIN-PRO ADMINISTRATION
Figure 6.12: Active Streams
6.8.1 Definition of an Active Stream
A stream is considered “active” if it has at least one message in the quarantine (pending, spam or non-spam) or has any rules, blocks or allow rules defined.
6.8.2 The Active Stream Display
The columns in the display are:
Stream The name of the stream. Each stream name is a hyperlink; if you click on the link, you will switch streams to that stream.
Pending The number of pending messages in the stream’s quarantine.
Spam The number of spam messages in the stream’s quarantine.
Non-Spam The number of non-spam messages in the stream’s quarantine.
Opted-In? Set to Yes if the stream is both approved for anti-spam scanning and opted-in; set to No otherwise.
Delete A column of links for deleting streams.
If you enter a string in the “Filter:” box, then CanIt-Domain-PRO limits the display to entries whose Stream column contains that string.
CanIt-Domain-PRO — AppRiver, LLC 6.9. FILTERING OUTBOUND MAIL 125
6.8.3 Deleting a Stream
To delete a stream, click on the Delete link in the Active Streams page. Then click on Yes, delete it! to confirm deletion. Deleting a stream deletes all incidents, rules, settings, etc. associated with the stream.
6.9 Filtering Outbound Mail
Some organizations like to add boilerplate disclaimers to outbound mail. CanIt-Domain-PRO can achieve this by streaming all outbound mail to an “outbound” stream, and adding boilerplate options for that stream. One way to stream all outbound mail to a particular stream is to set up your domain mappings as follows:
• All of your own domains (that is, domains considered “internal”) should have mappings set up. The mappings could be ChopDomain, Sendmail, or whatever, as long as the mappings exist.
• The wild-card domain * should have a domain mapping of Database.
• The wild-card address * should have an address mapping mapping it to the stream outbound. (You can name your outbound stream however you like.)
With these settings, mail for internal recipients will be streamed appropriately, and mail for external recipients will all be streamed to outbound. For the outbound stream, enter the appropriate boilerplate to add to outbound messages. You can also add custom body-matching rules if you want to quarantine mail containing certain words—for exam- ple, “Do Not Distribute Externally” Such rules on an outbound stream may help prevent unauthorized distribution of confidential information. See also Known Networks (Section 5.7 on page 65) for another way to force outbound mail into a specific stream. Using Known Networks may be simpler than using address mappings if all your outbound mail originates from a limited set of IP addresses.
6.9.1 DKIM-Signing Outbound Mail
DKIM, or DomainKeys Identified Mail is a mechanism for proving that a particular organization’s servers relayed a mail message. More specifically, DKIM uses cryptographic techniques that allow recipients to validate that a specific domain is indeed associated with the message. CanIt-Domain-PRO permits you to DKIM-sign outbound mail. Please note that the only way CanIt- Domain-PRO can validate the origin of a message is to look at the sending IP address. Therefore, CanIt-Domain-PRO DKIM-signs messages based on a domain being associated with a Known Net- works entry with the “Force-to-Stream” parameter (indicating outbound mail) set. For example, consider this Known Networks entry:
CanIt-Domain-PRO — AppRiver, LLC 126 CHAPTER 6. CANIT-DOMAIN-PRO ADMINISTRATION
Figure 6.13: Known Network with Associated Domains
In this example, CanIt-Domain-PRO will permit DKIM signing of a message:
• That has a From: header sender in the domain example.com or example2.net • Providing also that the message originates from 192.168.7.88/32
In order to DKIM-sign a message, CanIt-Domain-PRO requires a key pair to be generated. A key pair consists of two cryptographic keys that work together: The private key is a very large number that is kept secret. It is used to sign a message. The public key is another very large number that is connected to the private key and must be made public; anyone who has the public key can verify whether or not a message was indeed signed by the private key. The mathematics of signing is such that although possession of the public key permits verification that a message is signed, only possession of the private key permits the actual signing process. Additionally, it is believed that it is infeasible to derive the private key given only the public key. A DKIM public key is typically published by creating a special TXT DNS record that contains the public key and a few ancillary pieces of information required by DKIM verification software.
Managing DKIM Keys
To DKIM-sign a message from a particular domain, CanIt-Domain-PRO needs a key pair. To generate a key pair, click on Setup and then DKIM Keys. The DKIM Key List screen appears:
Figure 6.14: DKIM Key list
CanIt-Domain-PRO — AppRiver, LLC 6.9. FILTERING OUTBOUND MAIL 127
In Figure 6.14:
• The domain example.com has a single DKIM key pair. The DKIM selector is canit and the key is active, meaning it will be used to sign outbound mail. • The domain example.net has a single DKIM key pair whose selector is sel1. That key is not active, so it will not be used to sign outbound mail. Since example.net has no active keys, its outbound mail will not be signed at all. • The domain example.org has two DKIM key pairs: One with selector canit and another with selector canit2. The canit key is active, so it will be used for signing.
To add a new DKIM key pair, click on Add New DKIM Key Pair. The DKIM Key Pair page appears:
Figure 6.15: Adding a DKIM Key Pair
Enter a domain name and click Save. Note that the domain you enter must be associated with at least on Known Networks entry. Additionally, the domain must be within the current realm. The DKIM selector defaults to “canit”, but you can use any selector you like as long as it is at most 16 characters long and can appear as a legal domain name component. Once you have added the key pair, CanIt-Domain-PRO will display information about the key:
Figure 6.16: DKIM Key Details
The information displayed includes the domain name DNS TXT record required to publish the public key. Please note: DKIM TXT keys are typically quite long. We display them in BIND 9 multi-part
CanIt-Domain-PRO — AppRiver, LLC 128 CHAPTER 6. CANIT-DOMAIN-PRO ADMINISTRATION
string format. Different DNS software might require the record to be entered in a different format; in reality, the entire record is one long piece of text. Please consult your DNS provider’s documentation for more information on the format required for DKIM keys.
Activating a DKIM Key Pair
When you first create a DKIM key pair, it is not active. To activate the key pair, return to the DKIM key list and enable the Active radio button. Then click Submit Changes. If the domain has any other DKIM key pairs, they are automatically deactivated.
Deleting a DKIM Key Pair
To delete DKIM key pairs, click on Setup and then DKIM Keys. Enable the appropriate checkboxes in the Delete? column and click Submit Changes.
DKIM Selectors
CanIt-Domain-PRO allows you to specify a DKIM selector, but note that any given domain is only allowed to have one active DKIM key. DKIM selectors are useful should you wish to roll over your keys. Here is an example:
• Suppose you create a DKIM key pair with the selector s201501. CanIt-Domain-PRO creates the key pair and you publish a DNS record.
• Sometime later, you want to change the key pair because it’s good practice to change keys every now and then. Within CanIt-Domain-PRO, create a new key pair with a selector of (for example) s201506. Leave the old record for 201501 in place on your DNS server and publish an additional DNS record for the new s201506 key pair. To roll over to the new key, simply make it active; the old key will automatically be deactivated.
In this way, old messages can still be verified for as long as you keep the s201501 DNS record in place, but all new messages will be signed and verified with the new key pair and the selector s201506.
Selectively DKIM-Signing Outbound Mail
Sometimes, an organization may not wish to DKIM-sign all of its outbound mail. CanIt-Domain-PRO lets you selectively sign outbound mail as follows: When CanIt-Domain-PRO sees an outbound message, it computes the stream that the From: header address would be in. If, for some reason, it cannot find the stream, it uses the default stream in the realm of the From: header address. If the Quarantine Setting S-1050 Enable DKIM Signing for outbound messages originating from senders in this stream is set to Yes in the stream determined above, CanIt-Domain-PRO DKIM-signs
CanIt-Domain-PRO — AppRiver, LLC 6.10. COPYING RULES FROM ONE STREAM TO ANOTHER 129
the message. Otherwise, it does not. This allows you to avoid DKIM-signing bulk messages, auto- mated messages, etc. providing they originate from addresses with their own streams. By default, S-1050 is set to Yes, so by default outbound mail is DKIM-signed if a key pair is present and the messages comes from an associated domain of a Known Network.
A Note on some DKIM-Signing Pitfalls
CanIt-Domain-PRO uses software called Sendmail to actually accept and deliver messages. Send- mail may make its own header modifications to messages without CanIt-Domain-PRO’s knowledge, thereby breaking DKIM signatures. In most cases, programs used to compose email messages do so in such a way that Sendmail does not need to modify anything and DKIM works fine. But we recommend testing DKIM with all the mail software your users employ to ensure it generates correct signatures. In particular, Sendmail will change a header that looks like this: From: Full Name
6.10 Copying Rules from One Stream to Another
Occasionally, it is useful to copy or move rules from one stream to another. To do this, click on Administration and then Copy Rules. The Copy Rules page appears:
Figure 6.17: Copying Rules
To copy rules:
1. Choose which rules you wish to copy by activating the appropriate check boxes under Objects
CanIt-Domain-PRO — AppRiver, LLC 130 CHAPTER 6. CANIT-DOMAIN-PRO ADMINISTRATION
to Migrate.
2. Put the name of the stream you want to copy from in the From stream: box.
3. Put the name of the stream you want to copy to in the To stream: box.
4. Select “Preserve Original” or “Overwrite” to handle the case of conflicting rules in the source and destination streams.
5. Click on Copy Rules to copy rules from the source stream to the destination stream. Move Rules is similar, but any rule that is successfully placed in the destination stream is deleted from the source stream.
6.11 Secondary MX Hosts
Secondary MX hosts require special handling by CanIt-Domain-PRO. Secondary MX hosts which re- lay to the CanIt-Domain-PRO system should always be listed in “Known Networks”, with the options below checked, as it is usually desirable to modify CanIt-Domain-PRO behaviour as follows: Note that localhost (127.0.0.1) is always considered a secondary MX host for the purposes below:
Friendly Host When checked, rejected mail is simply discarded rather than being failed with a 5xx code. This prevents the friendly host from generating backscatter.
Parse Received Headers When checked, CanIt-Domain-PRO trusts the Received: header added by that connecting host or network. This means that CanIt-Domain-PRO will be able to apply host checks against the host that submitted the message to your network, rather than against your secondary MX server.
Prohibit Block Rules When checked, CanIt-Domain-PRO ignores any host blocks for hosts in this network. This will prevent locally-generated mail from your secondary MX hosts from being blocked. Note that if “Parse Received Headers” is enabled, mail relayed via the secondary system will show as being from the upstream IP, and blocks will not be ignored.
Skip RBL Lookups When checked, CanIt-Domain-PRO will suppress DNS blocklist lookups.
Skip Greylisting When checked, CanIt-Domain-PRO will suppress first-time sender checks.
Any machine under your control that you expect to forward mail to your machine should be considered a secondary MX host. For example, if a number of users have accounts on a machine that forward mail to your machine using .forward files, you should consider entering that machine as a secondary MX host. Also, note that if CanIt-Domain-PRO is able to determine the “real” relay IP by parsing the Received: headers, and you have enabled this option, then CanIt-Domain-PRO runs all the host checks as usual, using the real relay IP address. However, these checks are (of necessity) delayed until after the DATA phase of the SMTP transaction, because CanIt-Domain-PRO does not have the required information at the MAIL FROM: or RCPT TO: phases.
CanIt-Domain-PRO — AppRiver, LLC 6.12. AVOIDING BACKSCATTER 131
6.12 Avoiding Backscatter
Under most circumstances, if CanIt-Domain-PRO rejects a message, it responds with an SMTP failure code. This generally causes the sending relay to mail a failure notification to the original sender. However, because most spam and viruses have faked sender addresses, you may not want this behavior for messages relayed from a secondary MX host or for messages split into multiple streams. That’s because if a message is rejected after having been accepted by one of your mail servers, it’s the responsibility of the sending server to generate a failure Delivery Status Notification or DSN. If (as is likely) the sender address is faked, that failure message may arrive at an unsuspecting third- party. This is what is known as backscatter. It is a violation of RFC 821, and is generally considered bad behavior, to silently discard mail; how- ever, many sites are beginning to lump hosts responsible for generating backscatter into the same category as spammers. Because of this, CanIt-Domain-PRO will not generate a failure notification for mail from local host or from a designated secondary MX host.
6.13 Test Plugins
Some anti-spam tests are very specific and are implemented as plugins. Currently, CanIt-Domain-PRO ships with a number of plugins that are described in subsequent sections. If a plugin matches against a particular message, the plugin is said to have fired. To configure test plugins, click Rules and then Plugins. The Test Plugins page appears:
Figure 6.18: Test Plugins
For each plugin, you can configure actions to be taken on a per-stream basis (although we recommend creating rules only in the default stream.) To configure a plugin:
1. Select the action to be taken if the plugin fires. The action can be one of:
CanIt-Domain-PRO — AppRiver, LLC 132 CHAPTER 6. CANIT-DOMAIN-PRO ADMINISTRATION
• Ignore — do not use this plugin at all. • Hold/Tag — hold mail in the quarantine if the plugin fires. (In a tag-only stream, this will be converted to a tag.) • Score — add the score in the Score column if the plugin fires. • Reject — reject the mail if the plugin fires. 2. If you chose Score, enter a decimal score in the Score column. 3. If you wish, you can add a comment in the Comment column 4. Click Submit Changes to make the changes take effect.
6.13.1 The PhishingAddress Plugin
The PhishingAddress plugin consults a dynamically-updated list of e-mail addresses known to be used in phishing attacks. We recommend configuring it as follows:
• In the default stream, configure the test to add 10 points to the message score. Alternatively, you may wish to configure it to reject mail. • If you are routing outbound mail through CanIt-Domain-PRO, then you should be sending out- bound mail through a dedicated outbound stream. In that stream, configure the PhishingAddress plugin to reject mail. If users accidentally reply to a phishing e-mail that somehow got through, at least by rejecting their replies you will prevent sensitive information from reaching the at- tackers.
6.13.2 The PhishingURL Plugin
The PhishingURL plugin consults a dynamically-updated list of URLs known to be used in phishing attacks. It fires if a message contains one or more URLs on the list. We recommend configuring the test to add 10 points to the message score. Alternatively, you may wish to configure it to reject mail.
6.13.3 The OfficeMacros Plugin
The OfficeMacros plugin examines Microsoft Office attachments and fires if they contain macros. Since office documents containing macros can be extremely dangerous and can be used to spread malware and ransomware, we recommend scoring this plugin at 3.5. However, if you find there are too many false-positives, cautiously lower the score.
6.13.4 The OfficeMacro* Open Plugins
Three plugins named OfficeMacroAuto Open, OfficeMacroDocument Open and OfficeMacroWork- book Open fire if a Microsoft Office document contains a macro with the name Auto Open, Docu- ment Open or Workbook Open, respectively. In addition, the Auto Open plugin fires if the Microsoft Office document appears to invoke powershell.exe, cmd.exe or shell.exe.
CanIt-Domain-PRO — AppRiver, LLC 6.14. EMERGENCY BLOCKING OF DELIVERY STATUS NOTIFICATIONS 133
These macros are often used by malicious software to launch a virus payload. The default and recom- mended action for each plugin is to score 10 points. Note that legitimate spreadsheets fairly frequently contain the Workbook Open macro, so you may cautiously reduce the score for OfficeMacroWorkbook Open, although we recommend doing it on a case-by-case basis (rather than in the default stream) to limit the risk.
6.13.5 The Shortener404 Plugin
The Shortener404 plugin fires if an email contains a URL on a known URL shortener such as t.co, bit.ly, tinyurl.com, etc. that returns a 404 “Page Not Found” response code. This plugin can only score, not reject. Additionally, the score changes the behavior of the plugin:
•A negative score causes the plugin to be completely ignored; it is not run at all.
•A zero score causes the plugin to run. Any expanded URLs returned by the URL shortening services are added to the list of URLs to check against the Known Phishing URL database.
•A positive score causes the plugin to run; additionally, if any shortened URL returns a 404 response code, then the score is added to the incident’s score.
6.13.6 The NewlySeenDomain Plugin
The NewlySeenDomain plugin fires if the envelope sender of a message is from a domain that CanIt- Domain-PRO first saw less than 7 days ago. This plugin is designed to treat new domains with some degree of skepticism; we recommend scoring 1 point for newly-seen domains.
6.14 Emergency Blocking of Delivery Status Notifications
Sometimes, a spammer will process a large spam run and fake the sender address to be within a domain you control. Faking the sender address as if it comes from an innocent third-party is called a joe-job. Unfortunately, in a typical spam run, a large percentage of the recipient addresses are invalid, so the run creates many delivery failure notifications (officially called Delivery Status Notifications or DSNs). Because of the faked sender address, all of these notifications come back to you, the innocent third- party. These spurious failure notifications are called backscatter and can cause a huge load on your CanIt-Domain-PRO scanners, as well as a huge annoyance for end-users. CanIt-Domain-PRO has a feature that allows you to block DSNs for selected domains for a limited time. This is an emergency measure and should only be used for a limited time in the face of large amounts of backscatter. Normally, this feature is disabled. To enable the feature, click on Setup : Features and enable the Permit Emergency Blocking of Delivery Status Notifications feature. Next, click on Rules : Block DSNs. The Block Delivery Status Notifications page appears:
CanIt-Domain-PRO — AppRiver, LLC 134 CHAPTER 6. CANIT-DOMAIN-PRO ADMINISTRATION
Figure 6.19: Block Delivery Status Notifications Page
To turn on DSN-blocking for a domain:
1. Enter the domain name in the Domain box. 2. Pick an expiry date. The default expiry date is 5 days in the future. CanIt-Domain-PRO will not let you pick an expiry date more than 30 days in the future. 3. If you wish, add a comment explaining why you are enabling DSN-blocking. 4. Click Submit Changes
To edit the expiry date and comment for existing entries, change the text in the appropriate boxes and click Submit Changes To remove DSN-blocking from a domain, enable the appropriate Delete? checkbox and click Sub- mit Changes.
Note: DSN blocking applies to all streams in the realm. In this respect, it is different from other entries in the Rules menu which apply to a particular stream.
6.15 Removing All Rules and Settings from a Stream
On occasion, it may be necessary to delete all rules, blocks, allow rules, settings, etc. from a stream. If a novice user has created many such rules and settings, the stream may be unusable and a “factory reset” advised. To delete all rules and settings from a stream:
1. Switch to the stream in question with the View This Stream button. 2. Click on Rules. 3. Click on the link after the phrase “To delete all rules and stream settings for stream streamname, click here.” 4. Click Purge Rules to delete all the rules and settings, or Cancel to cancel.
Note: It is not possible to purge all rules from the default stream.
CanIt-Domain-PRO — AppRiver, LLC 6.16. PROVISIONING INFORMATION 135
6.16 Provisioning Information
CanIt-Domain-PRO keeps track of all the addresses and streams that have received mail in the last 30 days. It can display this information so you can track the usage of the system. To view provisioning information, click on Administration and then Provisioning. The Provisioning page appears:
Figure 6.20: Provisioning Information
The rather large provisioning table contains a number of columns. The columns are as follows:
• Realm - the name of the realm. The realm tree starting at the current realm is displayed along with little arrows to indicate the hierarchical structure. Realm names are links which, if clicked, display provisioning rooted at that realm.
• Domains - a list of domains mapped to the realm. Each domain name is followed by a green checkmark and the green letters “MX” if its MX records point to CanIt-Domain-PRO. If the MX records do not point to CanIt-Domain-PRO, then the domain name is followed by a red X and the red letters “MX”. Note that MX records are checked once a night by the nightly cron job, so the information displayed here may be slightly out of date. If a domain does not correctly validate recipients, the MX indicator is followed by a yellow hazard sign warning of the problem. Note that provisioning information for non-validating domains will not be accurate.
• Expiry - the expiry date (if any) associated with the realm.
• Addresses This Realm Only - the number of addresses in the realm that have received email in the last 30 days.
• Addresses Including Subrealms - the number of addresses in the realm and all of its descen- dants that have received email in the last 30 days.
• Streams This Realm Only - the number of addresses in the realm that have received email in the last 30 days.
• Streams Including Subrealms - the number of addresses in the realm and all of its descendants that have received email in the last 30 days.
CanIt-Domain-PRO — AppRiver, LLC 136 CHAPTER 6. CANIT-DOMAIN-PRO ADMINISTRATION
• Outbound Addresses This Realm Only - the number of outbound addresses in the realm. A realm is considered to be using outbound filtering if any of its domains is associated with a Known Networks entry (Section 5.7.) In this case, all of its inbound addresses are counted in the outbound column. Otherwise, no addresses are counted in the outbound column.
• Outbound Addresses Including Subrealms - the number of outbound addresses in the realm and all of its descendants.
• Outbound Streams This Realm Only - the number of outbound streams in the realm. The criteria and counting rules for outbound streams are similar to those for outbound addresses.
• Outbound Streams Including Subrealms - the number of outbound streams in the realm and all of its descendants.
If the Archiving add-on is installed, the following columns are present:
• Archiving Streams This Realm Only - the number of streams in the realm that have archiving enabled.
• Archiving Streams Including Subrealms - the number of streams in the realm and its descen- dants that have archiving enabled. This item is formatted as a list of count/retention pairs. For example, if a realm and its descendants have 45 streams archiving for 12 months, 201 archiving for 24 months and 16 archiving for 36 months, then the output will be: 45/12, 201/24, 16/36
• Archive Retention Months - the number of months for which archived mail is retained. This can be set on a realm-by-realm basis.
If the Secure Messaging add-on is installed, the following columns are present:
• Secure Messaging Streams This Realm Only - the number of streams in the realm that have secure messaging enabled.
• Secure Messaging Streams Including Subrealms - the number of streams in the realm and its descendants that have secure messaging enabled.
6.16.1 Computer-Readable Provisioning Information
To download the provisioning data in CSV format (suitable for importing into a spreadsheet), click the Download in CSV Format link at the bottom of the page. To download the data as JSON (suitable for processing by many scripting languages), click the Down- load in JSON Format link at the bottom of the page.
CanIt-Domain-PRO — AppRiver, LLC Chapter 7
External Authentication
7.1 Introduction
In addition to its built-in user list, CanIt-Domain-PRO can authenticate users using external mecha- nisms. To enable the use of external authentication mechanisms, these basic steps must be followed:
1.A User Lookup must be defined. A User Lookup describes to CanIt-Domain-PRO how to look up user information from an external source.
2. An Authentication Mapping must be created. An Authentication Mapping tells CanIt-Domain- PRO which User Lookup to use for a given domain. You can use different authentication mech- anisms for different domains, which gives CanIt-Domain-PRO considerable flexibility.
Some User Lookups can also perform streaming. That is, given an email address, they can return the name of the stream associated with that email address. The LDAP (Section 7.2.2) and Program (Section 7.2.4) User Lookups can perform streaming. Using a User Lookup to perform streaming is very powerful; for example, you could use an LDAP lookup to stream all of a user’s aliases into his single stream. CanIt-Domain-PRO also supports integration with Microsoft’s Azure Active Directory.
7.2 User Lookups
To create a User Lookup:
• Click on Setup and then User Lookups. You will see the User Lookup list:
CanIt-Domain-PRO — AppRiver, LLC 137 138 CHAPTER 7. EXTERNAL AUTHENTICATION
Figure 7.1: User Lookup List
• Click on Add a New User Lookup, and the User Lookup Wizard appears:
Figure 7.2: User Lookup Wizard
• Pick a name for the User Lookup, and click Next. The User Lookup method selection screen appears:
Figure 7.3: User Lookup: Method Selection
• Enter a comment for the lookup method. The comment can be anything you like; its purpose is to document the method so you remember what it does.
CanIt-Domain-PRO — AppRiver, LLC 7.2. USER LOOKUPS 139
• Select a lookup method. CanIt-Domain-PRO supports the following methods:
– POP3: CanIt-Domain-PRO authenticates users against a POP3 server. – IMAP: CanIt-Domain-PRO authenticates users against an IMAP server. – LDAP: CanIt-Domain-PRO authenticates users against an LDAP server. If you are creat- ing a new user lookup, then the LDAP choice is broken into four possibilities. The first two are appropriate if you are authenticating against Active Directory and the last two are appropriate if you are authenticating against a generic UNIX LDAP server: 1. LDAP (Active Directory: Log in using Windows username @ domain): This choice pre-fills settings that are suitable for logging in using your Windows user-name or Windows user-name followed by @ and the domain name. 2. LDAP (Active Directory: Log in using email address): This choice pre-fills settings that are suitable for logging in with your email address. 3. LDAP (Generic: Log in using username @ domain): This choice pre-fills settings that are suitable for logging in with your user-id (or user-id followed by @ and domain name.) 4. LDAP (Generic: Log in using email address): This choice pre-fills settings that are suitable for logging in with your email address.
Note: Once an LDAP user lookup is created, editing it shows the method as simply LDAP. The four possibilities enumerated above are simply conveniences that pre-select appropriate settings when you first create the user lookup. – Azure Active Directory integrates with Microsoft’s cloud-based Azure Active Directory. Note that you can use Azure Active Directory for streaming only, and not for authentica- tion. – Program: CanIt-Domain-PRO invokes a program (that you supply) to perform authenti- cation. – Program (Legacy method): CanIt-Domain-PRO invokes external programs in the same way as older versions did (using the “Alternate Authentication” global setting that has since been removed.) – Rewrite: This method cannot be used for authentication; it can only be used for stream mapping. It generates a stream name using simple rewriting rules on the email address.
• Normally, User Lookups may only be used by domains within the realm in which the User Lookup is defined. However, if you set “Allow subrealms to use this User Lookup?” to Yes, then domains in subrealms will be able to use the User Lookup. This is useful, for example, if you have a number of customer realms that are all back-ended on the same LDAP or IMAP server.
• Click Next.
7.2.1 IMAP and POP3 Authentication
If you selected IMAP or POP3 authentication methods, then the wizard looks like this:
CanIt-Domain-PRO — AppRiver, LLC 140 CHAPTER 7. EXTERNAL AUTHENTICATION
Figure 7.4: IMAP/POP3 User Lookup
To complete the setup:
• Enter the IP address or fully-qualified host name of the IMAP or POP3 server. If the server is listening on a non-standard port, add a slash followed by the port number to the server name. For example, if you have an IMAP server listening on port 1143 on the host magnesium, you could enter magnesium/1143 as the server.
• If you would like to strip the domain name from the login name before attempting authen- tication, set the “Strip domain name” setting to Yes. If someone logs in to CanIt-Domain- PRO as [email protected] and this setting is Yes, then the username passed to the IMAP or POP3 server is simply user. (The default home stream, however, is normally the full [email protected].)
• If you would like to strip the domain name from the home stream, set “Strip domain name from home stream after authentication?” to Yes. This means that if someone logs in as [email protected], her home stream will be user.
• If you would like CanIt-Domain-PRO to force user-names authenticated by POP3 or IMAP to lower-case, set “Force user name to lower-case” to Yes. (This also implicitly sets the home stream name on login to lower-case.) The user name is lower-cased before being presented to the POP3 or IMAP server.
• If you would like CanIt-Domain-PRO to force stream names (as determined by the POP3 or IMAP username) to lower-case, set “Force stream name to lower-case?” to Yes. If you want to preserve mixed-case stream names, set this setting to No (which is the default.)
• If you want CanIt-Domain-PRO to validate the SSL certificate of the server (assuming SSL or TLS is used), set “Validate server certificate” to Yes.
• Pick the appropriate encryption settings for CanIt-Domain-PRO to use when communicating with the POP3 or IMAP server.
CanIt-Domain-PRO — AppRiver, LLC 7.2. USER LOOKUPS 141
• By default, when a user successfully logs in via POP3 or IMAP, CanIt-Domain-PRO caches the username and encrypted password for 5 days. If your POP3 or IMAP server ever goes down, this permits users to continue to log in to CanIt-Domain-PRO (provided they have logged in successfully within the past 5 days.) You can change the cache time by editing “Number of days to cache successful credentials”. If you set this parameter to zero, then CanIt-Domain- PRO will not cache credentials upon successful login. • By default, a user logging in as [email protected] is put into the stream [email protected]. If you wish to rewrite the stream using a more sophisticated mech- anism than simply stripping the domain, enter a rewrite expression for “Rewrite expression to transform login name to stream name:” (Rewrite expressions are described in Section 7.2.7.) For example, suppose example.org and example.net are aliases. You want users to log in as either [email protected] or [email protected], but always want the stream to be [email protected]. In this case, use a Rewrite Expression of %[email protected]. • Click Next to see a summary of your settings. • If all of the settings are correct, click Finish to create the POP3 or IMAP User Lookup.
7.2.2 LDAP Authentication and Streaming
If you are creating a new user lookup, then the LDAP choice is broken into four possibilities. The first two are appropriate if you are authenticating against Active Directory and the last two are appropriate if you are authenticating against a generic UNIX LDAP server:
1. LDAP (Active Directory: Log in using Windows username @ domain): This choice pre-fills settings that are suitable for logging in using your Windows user-name or Windows user-name followed by @ and the domain name. 2. LDAP (Active Directory: Log in using email address): This choice pre-fills settings that are suitable for logging in with your email address. 3. LDAP (Generic: Log in using username @ domain): This choice pre-fills settings that are suitable for logging in with your user-id (or user-id followed by @ and domain name.) 4. LDAP (Generic: Log in using email address): This choice pre-fills settings that are suitable for logging in with your email address.
Note: Once an LDAP user lookup is created, editing it shows the method as simply LDAP. The four possi- bilities enumerated above are simply conveniences that pre-select appropriate settings when you first create the user lookup. LDAP user lookups can be used for one or both of user authentication and stream mapping. When used for stream mapping, the LDAP lookup method will also validate incoming email addresses against the LDAP server, allowing rejection of invalid recipients immediately at the CanIt gateway.
Note: In order for the LDAP User Lookup to validate incoming recipient addresses, it must be used for streaming in Domain Mapping. Be sure to use another method of validation (e.g. Verification Servers (see Section 5.4, Valid Recipients table) if you do not use your User Lookup for streaming.
CanIt-Domain-PRO — AppRiver, LLC 142 CHAPTER 7. EXTERNAL AUTHENTICATION
If you select one of the LDAP methods, you will see the LDAP User Lookup Wizard:
Figure 7.5: LDAP User Lookup
To complete the setup:
• In the “LDAP server(s)” box, enter the IP address or fully-qualified host name of your LDAP server. You can enter a comma-separated list of servers if you have more than one LDAP server. As with the IMAP and POP3 User Lookups, if a server listens on a non-standard port,
CanIt-Domain-PRO — AppRiver, LLC 7.2. USER LOOKUPS 143
enter a slash followed by the port number after the server name. For example, if you have two LDAP servers serverA and serverB, and the second listens on non-standard port 3389, enter the following into the server box: serverA, serverB/3389 If you want to use LDAPS (LDAP over SSL), enter the host name as an “ldaps” URL. For example: ldaps://server.example.com/
• Enter the Base DN of your LDAP tree in the “Base DN” box.
• Typically, CanIt-Domain-PRO needs to bind to the LDAP directory before it can search it. Enter the Bind DN in the “Bind DN” box. If a password is required, enter it in the “Bind password” box. Note that Active Directory does not support anonymous bind; a Bind DN and Bind password are required.
• If you wish to use this User Lookup for authentication, set “Use this method for authentication?” to Yes.
• If you would like to strip the domain name from the login name before attempting authentica- tion, set the “Strip domain name” setting to Yes. If someone logs in to CanIt-Domain-PRO as [email protected] and this setting is Yes, then the username passed to the LDAP server is simply user.
• If you would like CanIt-Domain-PRO to force user-names authenticated by LDAP to lower- case, set “Force user name to lower-case” to Yes. (This also implicitly sets the home stream name on login to lower-case.) The user name is lower-cased before being presented to the LDAP server.
• Enter the search filter for login authentication. The string %s will be replaced by the user’s login name. For most UNIX LDAP servers, a search filter of (uid=%s) is appropriate. For Active Directory, it might be (sAMAccountName=%s).
• To use the Locked Addresses feature, CanIt-Domain-PRO needs to know the e-mail address of a logged-in user. In most UNIX LDAP servers, this is stored in the mail attribute, while in many Active Directory servers, this is stored in the attribute proxyAddresses. Enter the appropriate value in “Attribute containing user’s e-mail address”.
• If you wish to control group membership using LDAP, enter the name of an LDAP attribute in the “Attribute containing group names” box. This attribute should contain a comma-separated list of group names. When a user authenticates, he/she will be considered to be a member of all of the groups listed in this attribute.
• By default, when a user successfully logs in via LDAP, CanIt-Domain-PRO caches the user- name and encrypted password for 5 days. If your LDAP server ever goes down, this permits users to continue to log in to CanIt-Domain-PRO (provided they have logged in successfully within the past 5 days.) You can change the cache time by editing “Number of days to cache successful credentials”. If you set this parameter to zero, then CanIt-Domain-PRO will not cache credentials upon successful login.
CanIt-Domain-PRO — AppRiver, LLC 144 CHAPTER 7. EXTERNAL AUTHENTICATION
• If you wish to use the LDAP server to stream addresses as well as authenticate, set “Use this method for streaming” to Yes.
• Enter the “Search filter for streaming”. For streaming, CanIt-Domain-PRO needs to look up an e-mail address in the LDAP server. For most UNIX servers, the appropriate search filter is (mail=%s), while for Active Directory, it is probably (proxyAddresses=smtp:%s). In the search filter, the string %s is replaced with the e-mail address. %u is replaced with the local part of the e-mail address (everything before ‘@’) and %d is replaced with the domain part of the address (everything after the ‘@’.)
• If you would like CanIt-Domain-PRO to force stream names (as determined by the LDAP lookup) to lower-case, set “Force stream name to lower-case?” to Yes. (This is the default.) If you want to preserve mixed-case stream names, set this setting to No.
• CanIt-Domain-PRO needs to know which LDAP attribute contains the stream name. For most UNIX servers, the appropriate attribute is uid, while for Active Directory, it is probably sAMAccountName. You can use a comma-separated list of attribute names for the “List of attributes to user for stream name” entry. CanIt-Domain-PRO will examine the attributes in order and set the stream name to the first attribute found that exists and is non-blank. This is useful if not all of your LDAP objects contain the same set of attributes, but they all contain at least one attribute appropriate for use as the stream name.
• If CanIt-Domain-PRO successfully looks up an e-mail address, but the LDAP record lacks an attribute for the stream name, CanIt-Domain-PRO can take one of the following actions:
– Tempfail the mail. We do not recommend this choice; it is available only for backward- compatibility with earlier versions of CanIt-Domain-PRO. – Place the mail in the default stream. – Place the mail in a stream whose name is the same as the entire email address. This is similar to AsIs address mapping. In this case, mail to [email protected] will go into a stream called [email protected]. – Place the mail in a stream whose name is the user-part of the email address. This is similar to ChopDomain address mapping. In this case, mail to [email protected] will go into a stream called user. – Place the mail in a stream whose name is the domain-part of the email address. This is similar to ChopUser address mapping. In this case, mail to [email protected] will go into a stream called example.org.
Set “Action if stream attribute missing” to the choice that is appropriate for your organization.
Note: Recipient Validation (i.e. rejecting SMTP RCPT with ”User Unknown” when the address is not found in LDAP) is only done if CanIt-Domain-PRO receives an actual response that there is no corresponding LDAP record for the given e-mail address. Changes to this setting do not affect validation.
Note: If the LDAP lookup for an email address returns more than one stream (because multiple LDAP entries match the address, for example), then CanIt-Domain-PRO picks a stream using the “Ac-
CanIt-Domain-PRO — AppRiver, LLC 7.2. USER LOOKUPS 145
tion if stream attribute missing” setting. It also raises an anomaly since this is usually a serious error in the LDAP data; a given email address should be owned by one and only one stream.
• Normally, CanIt-Domain-PRO tries the LDAP servers in order. If you would like it to try them in a random order (for load-balancing), set “Load-balance LDAP servers” to Yes.
• Some LDAP servers require CanIt-Domain-PRO to disconnect and reconnect and re-bind be- tween queries. (Active Directory requires this.) If your LDAP server requires this, set the “Reconnect for additional queries” setting to Yes.
• If you would like CanIt-Domain-PRO to cache stream lookups, set “Cache stream lookups in database” to Yes.
• You can change the connect timeout from the default value of 120 seconds to any value from 2 to 120 seconds. This timeout only applies to streaming lookups by the Perl filters. It does not apply to authentication, because PHP (used for the Web interface) does not have a way to specify an LDAP connect timeout.
Once you have entered the LDAP parameters, click Next to review your entries, and Finish to create the User Lookup.
7.2.3 Azure Active Directory Streaming
CanIt-Domain-PRO can perform directory lookups against Microsoft’s cloud-based Azure Active Di- rectory for the purpose of streaming. Integrating with Azure Active Directory requires setup steps to be performed both in Azure and in CanIt-Domain-PRO.
Configuration within Azure
To integrate CanIt-Domain-PRO with Azure Active Directory, log on to your Azure account. Then follow these steps:
CanIt-Domain-PRO — AppRiver, LLC 146 CHAPTER 7. EXTERNAL AUTHENTICATION
• Click on the Azure Active Directory item in the main menu. The Azure Active Directory screen pops up:
Figure 7.6: Azure Active Directory Main Screen
CanIt-Domain-PRO — AppRiver, LLC 7.2. USER LOOKUPS 147
• Click on App registrations and then create a new application. Use CanIt as the ap- plication name, Web app / API as the application type, and https://antispam. roaringpenguin.com/canit/ as the sign-on URL.
Figure 7.7: Azure Active Directory Application Registration
CanIt-Domain-PRO — AppRiver, LLC 148 CHAPTER 7. EXTERNAL AUTHENTICATION
• Once the application has been created, edit its settings. Copy the Application ID somewhere safe; you will need to enter it into CanIt-Domain-PRO at a later date. In the example screenshot, the application ID is cc9cca77-851e-4212-b015-07b1254560cb.
Figure 7.8: Azure Active Directory Application Settings
CanIt-Domain-PRO — AppRiver, LLC 7.2. USER LOOKUPS 149
• In the application settings screen, add API access for both the Windows Azure Active Di- rectory and Microsoft Graph APIs. In each case, select the appropriate API and then add permissions.
Figure 7.9: Azure API Access Settings
CanIt-Domain-PRO — AppRiver, LLC 150 CHAPTER 7. EXTERNAL AUTHENTICATION
• For each API that you add, enable the Read directory data permission in the Application Permissions section.
Figure 7.10: Azure Read Directory Permission
CanIt-Domain-PRO — AppRiver, LLC 7.2. USER LOOKUPS 151
• Go back to the Application Settings screen and create an API key. Use API as the description and Never expires for the expiration date. After you create the key, Azure will display the value of the key. Copy the key somewhere safe immediately. After this screen is refreshed, there will be no way to recover the key. The key will look like a random string of characters, something like this: hkvSAkukD7GsUFYZKP7MdoZL0gJLpA+xrFBWGSj+zHY=
Figure 7.11: Azure API Key
CanIt-Domain-PRO — AppRiver, LLC 152 CHAPTER 7. EXTERNAL AUTHENTICATION
Configuration within CanIt-Domain-PRO
Once you have created and configured the CanIt app within Azure and created an API key, you are ready to configure Azure withing CanIt. Run the User Lookup creation wizard and pick Azure Active Directory as the User Lookup Method. Click Next and the Azure Setup Page will appear:
Figure 7.12: Azure Setup within CanIt-Domain-PRO
Fill in the fields as follows:
• For Azure Tenant, use your Azure tenant name. This is typically the same as your domain name.
• For Application ID, use the Application ID you generated earlier within Azure.
• For Application Key, use the Application Key you generated earlier within Azure.
• The default Search Query looks for the login email address in the userPrinci- palName, the proxyAddresses and the otherMails fields. You may need to ad- just this query to suit your organization. The query syntax is described at https://msdn.microsoft.com/en-us/library/azure/ad/graph/howto/
CanIt-Domain-PRO — AppRiver, LLC 7.2. USER LOOKUPS 153
azure-ad-graph-api-supported-queries-filters-and-paging-options# filter. • The List of attributes to use for stream name is a comma-separated list of Azure AD at- tributes that are checked (in order) until a non-empty value is found. By default, we use the userPrincipalName value as the stream name. • The Action if stream attribute missing tells CanIt-Domain-PRO what do do if the search query finds an entry, but no stream attribute was found. This works exactly the same as with LDAP lookups described in Section 7.2.2. • You should NOT change anything in the Advanced Settings section. The default Mi- crosoft logon URL is https://login.windows.net and the default Graph API URL is https://graph.windows.net. Overriding those URLs is strictly for AppRiver test purposes.
7.2.4 Program Authentication and Streaming
With the Program User Lookup method, CanIt-Domain-PRO invokes an external program to authen- ticate users and map addresses to streams. If you select Program as your User Lookup type, the Program User Lookup Wizard appears:
Figure 7.13: Program User Lookup
To configure the Program User Lookup:
• Enter the full path to your “account-info” script. This is an executable script or program that you must supply. The path you supply must be an absolute path name. If you are running a CanIt-Domain-PRO cluster, this script must exist (and be identical!) on all scanning servers and the Web server. • If you would like to strip the domain name from the login name before attempting authenti- cation, set the “Strip domain name” setting to Yes. If someone logs in to CanIt-Domain-PRO as [email protected] and this setting is Yes, then the username passed to the program is simply user. The home stream, however, is normally [email protected]. • If you would like to strip the domain name from the home stream, set “Strip domain name from home stream after authentication?” to Yes. This means that if someone logs in as [email protected], her home stream will be user.
CanIt-Domain-PRO — AppRiver, LLC 154 CHAPTER 7. EXTERNAL AUTHENTICATION
• If you would like to cache stream lookups, set “Cache stream lookups in database?” to Yes. We strongly recommend enabling caching.
How the Program User Lookup is Invoked
• For authentication, the program is invoked as follows: /path/to/script --authenticate The program is then expected to read two lines from its standard input: The first line is a login name, and the second line is a password. The program must then validate the login name and password, and exit with one of the following exit codes:
–0 — Authentication was successful. –1 — Authentication failed.
• For obtaining user information, the program is invoked as follows: /path/to/script --info username Here, the program is passed the successfully logged-on user name as a command-line argument. It should print a series of key=value lines to its standard output, and exit with an exit status of 0. (The script doesn’t have to produce any output, but it can produce output if you want to pass extra information to CanIt-Domain-PRO.) The key/value pairs currently used by CanIt-Domain-PRO are:
– home stream=stream-name — sets the user’s home stream to stream-name instead of his or her login name. One possible use could be to convert a login name to all lower- case on systems that permit case-insensitive authentication. This ensures that no matter how the person logs in, she is directed to the correct stream name. – groups=group1,group2,...,groupN — when the user logs in, add her to all of the groups listed in the comma-separated list. – mail=email-address — set the user’s e-mail address to email-address.
• For mapping an e-mail address to a stream, the program is invoked as follows: /path/to/script --info-email address Here, address is an e-mail address that must be streamed. The script should write key=value lines to its standard output, and exit with one of the following exit codes:
–0 — the address exists and was successfully streamed. –1 — there was a temporary failure streaming the address. The mail will be tempfailed. – 67 — the address is not valid. CanIt-Domain-PRO will fail the SMTP RCPT command with a “User unknown” failure code.
If the address was streamed successfully, the script must print the following line to standard output: stream=stream-name
CanIt-Domain-PRO — AppRiver, LLC 7.2. USER LOOKUPS 155
This causes address to be mapped to stream-name. If no stream=stream-name line is emitted, but the script exits with a zero status, then CanIt-Domain-PRO falls back to database lookups, as described in Section 2.5 on page 33.
Sample Program for the Program User Lookup Method
The following is a very simple Bourne shell script illustrating how the Program User Lookup method works. Real scripts would obviously be more complex and probably written in a more appropriate language like Perl.
CanIt-Domain-PRO — AppRiver, LLC 156 CHAPTER 7. EXTERNAL AUTHENTICATION
#!/bin/sh do_auth () { read user read pass # In reality, we would do a directory lookup against LDAP or similar if test "$user" = "foo" -a "$pass" = "bar" ; then exit 0 fi exit 1 } do_info () { user="$1" # In reality, we would do a directory lookup against LDAP or similar if test "$user" = "foo" ; then echo "home_stream=foobar"; echo "[email protected]"; fi exit 0 } do_info_email () { email="$1" # In reality, we would do a directory lookup against LDAP or similar if test "$email" = "[email protected]" ; then echo "stream=foobar-stream"; fi if test "$email" = "[email protected]" ; then # No such user exit 67 fi exit 0 }
# Main program case "$1" in --authenticate) do_auth ;; --info) do_info "$2" ;; --info-email) do_info_email "$2" ;; *) exit 1; ;; esac
CanIt-Domain-PRO — AppRiver, LLC 7.2. USER LOOKUPS 157
7.2.5 Program Authentication (Legacy Method)
If you select this User Lookup method, then CanIt-Domain-PRO falls back to behavior compatible with previous versions. (This behavior is deprecated, however. New installations should use Program Authentication as described in Section 7.2.4.)
• If a program called /usr/share/canit/scripts/account-info exists and is exe- cutable, CanIt-Domain-PRO invokes it as if it were the script supplied for a Program User Lookup method.
• Otherwise, CanIt-Domain-PRO invokes /usr/share/canit/scripts/authenticate-user to authenticate users and /usr/share/canit/scripts/address-to-stream to convert an e-mail address to a stream. These scripts have been in use since CanIt-Domain-PRO 2.0 and are deprecated; you should convert to the new Program User Lookup method.
7.2.6 The account-info Script
Some User Lookup methods (such as POP3 or IMAP) as well as a lookup in the built-in user database are not capable of passing extra information back to CanIt-Domain-PRO. For that reason, if any User Lookup method other than Program or LDAP is used, CanIt-Domain-PRO still attempts to execute: /usr/share/canit/scripts/account-info --info username to obtain extra attributes (mail, groups and home stream) after a user logs in. If you need to set users’ e-mail addresses or home streams, but have them authenticate against an IMAP or POP3 server, simply supply an appropriate account-info script.
7.2.7 The Rewrite User Lookup
The rewrite user lookup is not used for authentication. It is only used to convert an address to a stream. It does so by rewriting the email address using a rewrite expression. To create a Rewrite User Lookup, enter the rewrite expression. CanIt-Domain-PRO rewrites an ad- dress as follows:
• The sequence %u in the rewrite expression is replaced with the local part of the email address (that is, everything before the @ sign.)
• The sequence %d in the rewrite expression is replaced with the domain part of the email address (that is, everything after the @ sign.)
• The sequence %s in the rewrite expression is replaced with the entire email address.
• Any other characters in the rewrite expression are copied as-is.
As an example of how you’d use the rewrite user lookup, consider an organization that owns the domains example.com, example.org and example.net. It wants any email address
CanIt-Domain-PRO — AppRiver, LLC 158 CHAPTER 7. EXTERNAL AUTHENTICATION
user@example.* to be placed in the stream [email protected]. That is, no matter what the domain on the incoming email address, it should be replaced with example.com. This can be accomplished by creating a rewrite user lookup with a rewrite expression of: %[email protected] and then using that user lookup as the Domain Mapping entry (Section 5.14) for all of the domains example.com, example.net and example.org Finally, observe that the Rewrite User Lookup can implement AsIs, ChopDomain and ChopUser streaming (Section 5.14). The relevant rewrite expressions are:
• AsIs: %u@%d
• ChopDomain: %u
• ChopUser: %d
7.3 Authentication Mappings
Once you have set up your User Lookup methods, you need to tell CanIt-Domain-PRO which method to invoke for each domain. To do this, click on Setup and then Authentication Mappings. The Authentication Mappings page appears:
Figure 7.14: Authentication Mappings
To create a new authentication mapping:
1. Enter the domain name in the Domain field. If you enter a single asterisk (“*”) in this field, then it is used as the default authentication mapping if an exact match is not found.
2. Select the User Lookup from the Mapping field. If there are any User Lookup methods added to ancestor realms of the current realm, they will appear as additional choices if they are marked as being available for subrealms.
CanIt-Domain-PRO — AppRiver, LLC 7.4. VIEWING CACHED LOGINS 159
3. Click on Submit Changes
In Figure 7.14, we see that anyone who logs in as [email protected] will be authen- ticated using the POP3-Sample User Lookup. Anyone logging in with a different domain (or no domain at all—simply user) will be authenticated using the LDAP-Sample User Lookup. If you enter a string in the “Filter:” box, then CanIt-Domain-PRO limits the display to entries whose Domain or Mapping columns contain that string.
7.4 Viewing Cached Logins
Some user-lookups cache successful logins. You can view the cache and delete entries from it by going to Administration and then Cached Logins. The Cached Logins screen appears:
Figure 7.15: Cached Logins
The screen displays all of the cached logins that CanIt-Domain-PRO is storing along with the date they will expire from the cache. Additionally, if CanIt-Domain-PRO was able to determine the user’s email address, that is displayed also. You can filter entries by entering a string to match in the filter box and clicking Filter. If people have left your organization and you want to remove their cached logins before they would normally expire, check the appropriate Delete? checkmarks and click Submit.
CanIt-Domain-PRO — AppRiver, LLC 160 CHAPTER 7. EXTERNAL AUTHENTICATION
CanIt-Domain-PRO — AppRiver, LLC Chapter 8
Bayesian Filtering
8.1 Introduction to Bayesian Filtering
Bayesian filtering is a statistical technique whereby CanIt-Domain-PRO assigns a spam probability based on training from users. Bayesian filtering can greatly improve the accuracy of CanIt-Domain- PRO, and makes it harder for spammers to evade filtering. Please consult the CanIt-Domain-PRO User’s Guide for additional details on using Bayesian filtering. This guide only contains information relevant when setting up and administering CanIt-Domain-PRO.
8.2 Unauthenticated Voting
Normally, to vote if a message is spam or not spam, a user must log in. You can configure CanIt- Domain-PRO to permit unauthenticated voting; this can make life easier for end-users who can just click on a link without worrying about entering a user name and password.
Note: Think carefully about permitting unauthenticated voting. If voting links ever escape your organization (as part of a forwarded message, for example), and your CanIt-Domain-PRO Web interface is exter- nally accessible, outsiders can cast votes. We strongly recommend permitting unauthenticated voting only if access the the CanIt-Domain-PRO Web interface is controlled in some other way. To permit unauthenticated voting:
• Under Preferences and Quarantine Settings, set Permit unauthenticated voting to Yes
You can permit unauthenticated voting on a stream-by-stream basis. If you permit it in the default stream, then it will be permitted in all streams that inherit from default (and that don’t override the setting.)
CanIt-Domain-PRO — AppRiver, LLC 161 162 CHAPTER 8. BAYESIAN FILTERING
8.3 The Bayes Journal
Bayesian training can be slow because it involves many database updates. For that reason, when you train a message, CanIt-Domain-PRO simply makes a note of the fact that the message is to be trained in a special table called the Bayes Journal. Periodically, a CanIt-Domain-PRO daemon process goes through the Bayes Journal and actually updates the Bayes data. For this reason, if you train some messages, these results will not immediately appear in the Bayes Settings page. The Bayes Journal is run every 10 minutes or so, so your training should appear within 10-15 minutes.
8.4 Site-Wide and Realm-Wide Bayes Training
Whenever someone hand-trains a message, the message is trained in the default stream of the realm as well as the stream containing the message. Additionally, it is trained in the default stream of all ancestor realms. For example, if the realm foo is a subrealm of base and the realm bar is a subrealm of foo, then hand-training a message in the stream bar:quux also trains it in bar:default, foo:default and base:default. You may wish to add some or all of these ancestor-realm default streams to the list of streams from which Bayes training is inherited.
8.5 RPTN
RPTN stands for the Roaring Penguin Training Network, and is a mechanism whereby multiple CanIt installations can share Bayes votes. RPTN contains two main parts:
1. In the reporting phase, CanIt-Domain-PRO installations send reports about whether or not mail they have seen is spam. A report essentially consists of a list of tokens in the mail message and a spam or not-spam flag, depending on how the incident was disposed of. The RPTN server aggregates all of the reports it receives and builds a database of Bayesian statistics from the reports.
2. In the download phase, a CanIt-Domain-PRO installation downloads the aggregated data and installs it in its database. This data can subsequently be used for Bayesian analysis.
To set up RPTN, click on Setup and then Wizards. Choose the RPTN Setup Wizard. The wizard leads you through the following steps:
1. You are asked if you would like to download Bayes data from RPTN.
2. If you answered Yes in Step 1, you are given an opportunity to limit when RPTN data is down- loaded. Downloading RPTN data can place a fair amount of load on the server, so you should limit RPTN downloads to off-peak hours. Be sure to leave at least a four-hour download win- dow, because RPTN checks are made every two hours. If the download window is too short, you may miss a download.
CanIt-Domain-PRO — AppRiver, LLC 8.6. RULESET AND GEOLOCATION DATA UPDATES 163
3. You are asked if you would like to submit reports to RPTN.
4. If you answered Yes in steps 1 or 3, you are prompted for your download username and pass- word. You cannot submit RPTN reports or download RPTN data unless you supply a valid username and password.
5. Your settings are summarized, and you are prompted to click Finish to save the changes.
RPTN data are downloaded into a stream called @@RPTN. If you would like to use RPTN data in Bayesian analysis, you must include @@RPTN in the stream setting “Inherit Bayes training history from these streams”. If you want all streams to inherit Bayes data from @@RPTN, then set the “Inherit Bayes training history from these streams” setting in the default stream in the base realm.
Note: To download RPTN data, the CanIt-Domain-PRO server must be able to make outgoing HTTPS con- nections (over TCP port 443) to the machine server.rptn.ca. To submit RPTN reports, the server must be able to make outgoing HTTPS connections to server.rptn.ca and also be per- mitted to send outgoing e-mail to [email protected]. If you have a firewall in front of the CanIt-Domain-PRO server, please ensure that the firewall rules permit the RPTN traffic.
8.6 Ruleset and Geolocation Data Updates
In addition to downloading Bayes data, CanIt-Domain-PRO uses your RPTN credentials to download two other sets of data:
• Updated rules that are pushed out from time-to-time by AppRiver.
• Geolocation data that maps IP addresses to countries and cities. (The data are derived from the GeoLite City data from MaxMind, which requires the following acknowledgement: This product includes GeoLite data created by MaxMind, available from http://www.maxmind.com/)
The updated rulesets are simply SpamAssassin rules that AppRiver publishes as required when a new spam variant is detected. The geolocation data is used by the country rules as described in the User’s Guide. CanIt-Domain-PRO also tokenizes the country, region, city and latitude/longitude of the sending relay for use in the Bayes database.
CanIt-Domain-PRO — AppRiver, LLC 164 CHAPTER 8. BAYESIAN FILTERING
CanIt-Domain-PRO — AppRiver, LLC Chapter 9
Permissions
9.1 Introduction
In addition to the fairly coarse-grained settings described in Section 6.5.1, “User Privileges”, CanIt- Domain-PRO allows you to implement fine-grained control over access to various parts of the Web- based interface. CanIt-Domain-PRO has two kinds of permissions:
1. Stream Permissions control access to CanIt-Domain-PRO features that affect the filtering of e- mail. For example, the ability to allow or block senders, create custom rules, and so on are all Stream Permissions. Stream Permissions depend on both the user and the stream; a given user may have different permissions in different streams.
2. User Permissions control access to various parts of the CanIt-Domain-PRO user-interface not directly connected to filtering mail. For example, access the different GUI preferences and the ability to do WHOIS lookups are all User Permissions.
CanIt-Domain-PRO can associate permissions with users and with groups. Any user can be a member of zero or more groups. CanIt-Domain-PRO always grants a user the union of all his user-specific permissions and all his group permissions. Adding a user to a group, therefore, can only ever grant additional permissions. It cannot take away permissions.
9.2 Stream Permissions
Every stream has associated with it an ordered list of stream classes. When CanIt-Domain-PRO looks up stream permissions, it first calculates the list of stream classes associated with a particular user and stream. Here is how CanIt-Domain-PRO computes the list of stream classes:
1. The name of the stream always comes first. Thus, for example, if you are viewing a stream called mystream, then the list of stream classes starts with mystream.
CanIt-Domain-PRO — AppRiver, LLC 165 166 CHAPTER 9. PERMISSIONS
2. If mystream happens to be your “home stream” (Section 4.6), then @@HOME is added to the list of stream classes.
3. If you have write-access in mystream, then @@WRITABLE is added to the list of stream classes.
4. If you have read-access in mystream, then @@READABLE is added to the list of stream classes.
5. Finally, the wildcard value * is added to the end of the list of stream classes.
When CanIt-Domain-PRO determines what permissions you have in a particular stream, it uses the following procedure:
1. It looks for permissions granted in the actual stream name. If it finds any, it stops searching the stream classes.
2. Otherwise, it checks the the stream classes and adds all permissions found to the set of granted permissions.
9.3 Determining Permissions
To determine a particular user’s permissions, CanIt-Domain-PRO performs the following steps:
1. First, it gathers all permissions associated with the particular user’s login ID. (These permissions are shown in Figures 9.3 and 9.4.)
2. Next, it adds all permissions granted to all the groups to which the user belongs.
3. If there was no entry in the permissions table for the particular user (that is, if Step 1 found no entries), then CanIt-Domain-PRO performs the following steps:
(a) If the user has root privileges, then CanIt-Domain-PRO adds all permissions granted to the pseudo-user *root* or *localroot* in the user’s realm. (b) Next, CanIt-Domain-PRO adds all permissions granted to the wild-card user * in the user’s realm.
4. If no entry was found for Step 3, then CanIt-Domain-PRO performs the following steps:
(a) If the user has root privileges, then CanIt-Domain-PRO adds all permissions granted to the pseudo-user *root* found in the first ancestor realm encountered on the path up to base. (b) Next, CanIt-Domain-PRO adds all permissions granted to the wild-card user * in the first ancestor realm encountered on the path up to base.
CanIt-Domain-PRO — AppRiver, LLC 9.4. GRANTING PERMISSIONS 167
9.4 Granting Permissions
To grant or deny permissions, click on Administration and then Permissions. The Permissions Page appears:
Figure 9.1: Permissions Page
If you enter a string in the “Filter:” box, then CanIt-Domain-PRO limits the display to entries whose User column contains that string. If you want to edit permissions for groups rather than users, click on the Groups link:
Figure 9.2: Permissions Page
9.4.1 Granting Stream Permissions
To grant stream permissions, click on the Edit link in the Stream Permissions column. The Stream Permissions page appears:
CanIt-Domain-PRO — AppRiver, LLC 168 CHAPTER 9. PERMISSIONS
Figure 9.3: Stream Permissions Page
• To enable a stream permission in a particular stream or stream class, enable the checkbox in the appropriate row and column.
• To enter the name of a stream or stream class, enter it into the text box in the Per-Stream Permission row. Note that when you enter permissions for a new user, you must enter the stream class in the text box, or your changes will be discarded.
• To delete all permissions for a particular stream or stream class, click the Delete link at the bottom of the appropriate column.
• To view permissions only for one stream or stream class, click on the stream or stream class name.
• To make your changes take effect, click Submit Changes.
The Stream Permissions are:
• Block Senders – The user is permitted to block senders.
• Always Allow Senders – The user is permitted to always-allow senders.
• Hold/Tag Senders – The user is permitted to add a hold rule for senders.
• Block/Always Allow/Hold/Tag Domains – These permissions are similar to the Sender Action permissions, but they apply to domain rules.
• Block/Always Allow/Hold/Tag Networks – These permissions are similar to the Sender Action permissions, but they apply to network rules.
• Reject/Accept/Hold/Tag MIME Types – These permissions are similar to the Sender Action permissions, but they apply to MIME type rules.
• Reject/Accept/Hold/Tag Filename Extensions – These permissions are similar to the Sender Action permissions, but they apply to filename extension rules.
CanIt-Domain-PRO — AppRiver, LLC 9.4. GRANTING PERMISSIONS 169
• Custom Rules – The user is permitted to create custom rules.
• SPF Rules – The user is permitted to create SPF rules.
• RBL Rules – The user is permitted to create RBL rules.
• Country Rules – The user is permitted to create country-code rules.
• Bayes Settings – The user is permitted to edit Bayes scoring rules.
• Blocked Recipients – The user can block recipients.
• Valid Recipients – The user can enter recipients into the Valid Recipients Table.
• See Pending/Non-Spam/Spam Message – The user can see the specified message type in the quarantine. Note that these permissions are normally off for @@READABLE streams; otherwise, the user could see default’s spam quarantine.
• Add Alternate Addresses to Streams – The user can add aliases to his/her stream.
• Opt In/Out – The user can opt in or out of spam-scanning.
• Adjust Notification Settings – The user can adjust his or her notification settings.
• See Per-Stream/Global Reports – The user can see the specified reports.
• Quarantine Settings – Every quarantine setting has an associated permission. The user can only see a quarantine setting if its corresponding permission has been granted. The user can only change a quarantine setting if the permission has been granted and the user has write- access in the stream.
Note: If a user does not have write-access in a stream, then permissions such as Custom Rules, Always Allow Senders, etc. merely permit the user to see the rules. He or she still cannot change them.
9.4.2 Granting User Permissions
To grant user permissions, click on the Edit link in the User Permissions column. The User Permis- sions page appears:
CanIt-Domain-PRO — AppRiver, LLC 170 CHAPTER 9. PERMISSIONS
Figure 9.4: User Permissions Page
The following User Permissions may be granted:
• Preferences – Unless this permission is granted, the user will not have access to the Preferences menu or any of its sub-menus.
• WHOIS Lookups – If this permission is granted, the user will be allowed to do WHOIS lookups.
• See Statistics – Allows the user to see the Reports : Statistics page.
• Use Log Searching – Allows the user to use the Log Searching feature (Chapter 17).
Note: Users must have root privileges to use Log Searching; non-root users cannot use it even if Use Log Searching is enabled. Also, the log-searching feature is available only on CanIt-Domain- PRO appliances.
• See User’s Guide – Enables the link to the user’s guide.
• Use API – Allows the user to access the REST-based CanIt-Domain-PRO API. See the API Guide for details.
• Provision Domains via API – Allows the user to provision new realms and domains via the API. Note that a user must be a realm administrator and must have API access to be able to provision domains. This option is available only on our CanIt-Domain-PRO appliances and Hosted CanIt.
Note: Allowing users to provision new realms and domains grants them tremendous power and may be a security risk. Do not grant this permission except to highly-trusted realm administrators
• Use Expert Interface – Grants the user access to the expert interface.
• Create RSS Feed – Grants the user permission to create an RSS feed link for pending messages.
CanIt-Domain-PRO — AppRiver, LLC 9.5. PERMISSION GRANTABILITY 171
• Turn off Stream Inheritance – Grants the user permission to completely isolate his stream by disabling inheritance from the default stream. We do not recommend granting this permission as a matter of course.
• Preferences – Each preference setting has an associated permission. A user can only change those settings for which permission has been granted.
9.5 Permission Grantability
In CanIt-Domain-PRO, the System Administrator always has all permissions and can grant or deny all permissions. However, the System Administrator can both limit the permissions available to Realm Administrators (as described in Section 9.4) and limit which permissions Realm Administrators can grant to themselves and other users. To modify which permissions realm administrators can grant, click on Administration and then Per- mission Grantability. The Grantable Permissions page appears:
Figure 9.5: Permission Grantability
To delete an entry from the Permission Grantability table, check the Delete checkbox and click Submit Changes. To edit which permissions a user can grant, click on the Edit link in the appropriate table row. To add a user to the table, enter the user ID in the User box and click Add. You can specify a realm in the Realm pulldown; if you do not, then CanIt-Domain-PRO will determine the realm based on the user ID. Whether you add a new user or edit an existing user, CanIt-Domain-PRO brings up the Grantable Permissions Detail page:
CanIt-Domain-PRO — AppRiver, LLC 172 CHAPTER 9. PERMISSIONS
Figure 9.6: Grantable Permissions Detail
To allow a user to grant a permission, check the appropriate checkbox. To prevent a user from granting a permission, uncheck the checkbox. Click Submit Changes when you have set permission grantabil- ity as you desire.
Note: If you prevent a user from granting a permission, you should also turn off that permission for the user. Otherwise, the user will lose the permission if he/she ever updates his permissions. For example, if a realm administrator is permitted to block senders, but not allowed to grant that permission, then if she ever modifies her own permissions, she will lose the “Block Sender” permission.
9.5.1 Grantability Algorithm
CanIt-Domain-PRO uses the following algorithm to determine which permissions a realm admin- istrator can grant. For the sake of illustration, assume that the realm administrator’s user ID is [email protected] and the realm name is myrealm.
1. First, CanIt-Domain-PRO looks for a grantability entry specifically for [email protected] in the realm myrealm. If it finds such an entry, it uses it.
2. If Step 1 found no entry, CanIt-Domain-PRO looks for an entry for the user * in the realm myrealm.
3. If Step 2 found no entry, CanIt-Domain-PRO looks for an entry for the user * in the realm base.
4. If Step 3 found no entry, then the realm administrator is allowed to grant any permission.
CanIt-Domain-PRO — AppRiver, LLC Chapter 10
Streams, Inheritance and the Simple GUI
10.1 Simplification
CanIt-Domain-PRO is extremely versatile, allowing end-users to set many parameters such as block rules, allow rules, custom rules, and so on. For many users, this is intimidating—the users may be unsophisticated, and just want to “make spam stop.” CanIt-Domain-PRO allows the administrator to set up special streams with pre-configured settings. Unsophisticated users then see a very simple interface which allows them to choose from one of these settings. CanIt-Domain-PRO achieves this with stream inheritance and special streams.
Note: Users who use the Simple GUI will not have their own quarantines. Special streams should be con- figured to pass, tag or reject. If any incidents are actually created, someone with administrative access will need to check the special streams’ quarantines periodically.
10.2 Stream Inheritance
Streams in CanIt-Domain-PRO inherit rules and settings from other streams. By default, all streams in a given realm inherit rules and settings from the default stream in that realm. The default stream, in turn, inherits rules and settings from the default stream in the parent realm and so on all the way up to the base realm. If a stream stream1 inherits from another stream stream2, we refer to stream2 as the parent of stream1. Conversely, we call stream1 the child of stream2. Furthermore, suppose that stream2 inherits from stream3. We then call stream3 and stream2 the ancestors of stream1. These terms are illustrated in Figure 10.1:
CanIt-Domain-PRO — AppRiver, LLC 173 174 CHAPTER 10. STREAMS, INHERITANCE AND THE SIMPLE GUI
stream3 stream2 inherits from stream3 parent
child stream2 stream1 inherits from stream2 stream1 is the child of stream2 parent stream2 is the parent of stream1 child stream3 and stream2 are the ancestors of stream1 stream1
Figure 10.1: Stream Inheritance Terminology
In addition to the default inheritance, streams can be configured to inherit rules and settings from Special Streams (discussed next in Section 10.3.) To determine a stream’s inheritance, CanIt-Domain-PRO consults the Stream Inheritance Table. To see this table, click on Administration and then Inheritance:
Figure 10.2: Stream Inheritance Table
To determine a stream’s parent, CanIt-Domain-PRO first looks up the stream in the inheritance table. If there is an entry, then that entry is used to determine the parent. If there was no entry, CanIt-Domain- PRO looks up the key “*” in the inheritance table. If such an entry exists, it is used to determine the parent. In the example in Figure 10.2:
• user3 inherits from 01 Tag Only. • user4 inherits from 00 Opt Out. • user5 does not inherit from any other stream. • user9 inherits from default.
CanIt-Domain-PRO — AppRiver, LLC 10.3. SPECIAL STREAMS 175
• All other streams (except for default) inherit from 01 Tag Only, because of the wildcard entry.
If you enter a string in the “Filter:” box, then CanIt-Domain-PRO limits the display to entries whose Stream or Inherits From columns contain that string.
10.3 Special Streams
A Special Stream is a normal stream with two extra behaviors:
• Other streams are allowed to inherit from special streams. Normally, a stream can only have default as its parent. If you add special streams, however, other streams are allowed to make the special streams their parents.
• If a stream inherits from a special stream, then mail for the child stream is held in the parent’s quarantine. That is, by inheriting from a special stream, a stream “loses” its quarantine, giving responsibility for any quarantined mail to the special stream.
10.3.1 Final Streams
A special stream may be marked final. If a special stream is marked final, then children of that stream may not override the special stream’s rules or settings. If a stream inherits from a final special stream, it’s as if the stream has given all control over to the special stream. To see special streams, click on Administration and then Special Streams. The Special Stream Table appears:
Figure 10.3: Special Stream Table
10.3.2 Creating Special Streams
To create a special stream, enter the name of the stream in the Stream text box, and a user-friendly description in the Description box. Then click Add Special Stream.
CanIt-Domain-PRO — AppRiver, LLC 176 CHAPTER 10. STREAMS, INHERITANCE AND THE SIMPLE GUI
In the example, the four streams 00 Opt Out, 10 Tag Only, 20 IT Staff and 30 Aggressive have been created. (Special streams are presented to end-users in order of the stream name, so we named the streams beginning with numbers so they would sort from least to most aggressive. We leave gaps between the stream numbers so we can insert more streams in between if required.) Once you have created the special streams, configure them appropriately. For example, for 00 Opt Out, you’d switch into that stream, and then under Preferences : Opt In/Out, you’d opt that stream out. (For convenience, you can click on a stream name in the Special Stream Table to switch into that stream.) For 30 Aggressive, you might change the stream settings to auto-discard anything scoring 8 or more on the spam scale. For 20 IT Staff, you could have CanIt-Domain- PRO hold suspect spam, and have a member of your IT staff check 20 IT Staff’s quarantine and release false-positives. Note that 00 Opt Out and 20 IT Staff are marked final. This means that rules and settings in streams inheriting from these two special streams are ignored; only the special streams’ settings and rules are used. On the other hand, streams inheriting from 10 Tag Only and 30 Aggressive may define their own rules, settings, block rules and allow rules. You can define as many special streams with as many different settings as you deem appropriate. Note that all special streams (by default) inherit from the default stream.
10.3.3 Deleting Special Streams
To delete a special stream, enable the checkbox in the Delete? column for the appropriate stream. Then click Submit Changes. Warning: If you delete a special stream, then all inheritances from that stream are deleted. Please see Section 10.2 for more details.
10.4 The Simplified GUI
If the CanIt-Domain-PRO administrator enabled the global setting G-4060 Users authenticated by alternate means default to simple GUI? (Section 6.1), then such users only see the Simplified Inter- face:
Figure 10.4: Simplified Interface
CanIt-Domain-PRO — AppRiver, LLC 10.5. INHERITANCE FROM NON-FINAL STREAMS 177
The simplified interface simply lists the possible Special Streams. The currently-inherited special stream is highlighted in bold red print. To inherit from a different stream, the user simply clicks on the appropriate radio button and clicks Set Spam-Scanning Level. This adjusts the entry in the inheritance table. To log out, the user clicks on Log Out. If the user clicks on Enable Expert Interface, then he or she will have access to the usual CanIt- Domain-PRO interface. He or she can then turn off inheritance (via Preferences : Set Default Stream) and take control over his or her own block and allow rules and spam quarantine.
Note: If you have set the global setting G-4075 Switching to expert mode cancels stream inheritance to Yes, then the act of clicking Enable Expert Interface cancels any inheritance that was in force, making the stream inherit from default again. To get back to the simple GUI, click on Simple Interface top-level menu entry. Note that this menu entry does not appear until at least one special stream has been defined.
10.5 Inheritance from Non-Final Streams
If a stream inherits from a non-final stream, CanIt-Domain-PRO uses the following procedures to resolve rules. In these examples, we assume that stream john inherits from the non-final stream 10 Tag Only
• For sender, domain and network block and allow rules, and for MIME type and Filename Ex- tension rules, CanIt-Domain-PRO first looks for a rule associated with the original stream (in our example, john.) If no such rule is found, it then tries the parent stream (in our example, 10 Tag Only) and then the parent of the parent, and so on up the inheritance chain.
• For custom rules, CanIt-Domain-PRO uses all the rules associated with the original stream in addition to rules associated with the ancestor streams.
• Bayes data is associated with the original stream (john) and not the parent stream (10 Tag Only).
10.6 Inheritance from Opted-Out Streams
If a stream or any of its ancestors is opted-out of spam-scanning, then no spam scanning is performed.
CanIt-Domain-PRO — AppRiver, LLC 178 CHAPTER 10. STREAMS, INHERITANCE AND THE SIMPLE GUI
CanIt-Domain-PRO — AppRiver, LLC Chapter 11
Periodic Reports
11.1 Introduction
CanIt-Domain-PRO can generate PDF reports about mail filtering activity and e-mail them to specified recipients.
11.1.1 Periodic Reports
A periodic report has a name, a page size, a recipient and a recurrence. The name can be anything you pick. The page size can be one of “US Letter” or “A4”. And the recipient can be any valid e-mail address. The recurrence specifies how often the report should be generated and mailed out. Possible choices for the recurrence are:
• On Demand — the report is never generated and mailed automatically, but only when specifi- cally requested from the Web interface. • Daily — the report is generated and mailed daily. • Weekly — the report is generated and mailed weekly. You can choose the day of the week. • Monthly — the report is generated and mailed monthly. You can choose either the first or fifteenth day of the month.
11.1.2 Charts
A chart produces a single PDF page in a periodic report. It contains a chart corresponding to a particular statistical query. A chart has a name (which can be anything you pick) and a type. The available chart types are described below. Note that all charts accept parameters that modify the results. For example, you can restrict the types of mail counted (you might only want to count spam, for example), the destination domains, etc. In addition to producing a page in the PDF report, each chart also generates a CSV file for importing into spreadsheet software. (Some charts only produce CSV files and no PDF output; if this is the case, it will be noted in the chart’s description.)
CanIt-Domain-PRO — AppRiver, LLC 179 180 CHAPTER 11. PERIODIC REPORTS
• Classification of Recent Mail. A pie chart showing the breakdown of recently-received e- mail. (“Recent” e-mail is defined by Global Setting G-1550, “Number of hours to keep detailed statistics”)
• Top Mail Countries. A pie chart showing the top countries sending recent e-mail.
• Top Domains. A pie chart showing the top recipient domains receiving recent e-mail.
• Top Mail Relays. A pie chart showing the top sending relays that have sent recent e-mail.
• Top Recipients. A pie chart showing the top recipient addresses receiving recent e-mail.
• Top Streams. A pie chart showing the top streams receiving recent e-mail.
• Top Viruses. A pie chart showing top recently-received viruses.
• Summary of Greylisting per Hour. A bar-chart showing how much recent e-mail was greylisted and ungreylisted.
• Summary of Mail per Hour. A bar-chart showing the classification of recent e-mail per hour.
• Classification of Long-Term Mail. A pie chart showing the breakdown of received e-mail over the long term. The timespan available in long-term statistics is determined by Global Setting G-1500, “Expire statistics after this many days”.
• Top Domains (Long-Term Statistics). A pie chart showing the top recipient domains over the long-term.
• Top Realms (Long-Term Statistics). A pie chart showing the top recipient realms over the long-term.
• Top Streams (Long-Term Statistics). A pie chart showing the top recipient streams over the long-term.
• Summary of Greylisting per Day. A bar chart showing how much mail was greylisted and ungreylisted over the long-term.
• Summary of Mail per Day. A bar chart showing how daily classification of mail over the long-term.
• Summary of Mail per Realm per Day. A bar chart showing daily mail volume per realm over the long term.
• Number of Email Addresses Seen by Realm. A chart showing the number of addresses seen in the last 30 days, broken down by realm. Note that this chart is only available as a CSV file; it does not produce PDF output.
CanIt-Domain-PRO — AppRiver, LLC 11.2. CREATING CHARTS 181
11.2 Creating Charts
The first step in creating a periodic report is to create one or more charts. Click on Reports : Periodic Reports. The main Periodic Reports page appears:
Figure 11.1: Periodic Reports
To add a chart:
1. Click Add a New Chart. 2. Enter a name for your chart. This name will appear as the page title in the final reports. 3. Select a chart type. 4. Click Next...
Once you have selected a chart type, CanIt-Domain-PRO will display a page for setting parameters for the chart. Set the parameters as appropriate for your chart and click Save Chart. To edit an existing chart’s parameters, click on its name in the Name column. To rename a chart, enter its new name in the Rename To... box and click Submit Changes. To delete a chart, enable the corresponding checkbutton in the Delete... column and click Submit Changes.
11.3 Creating Periodic Reports
Once you have created one or more charts, you can create periodic reports. To create a new periodic report, click Add a New Report. The Add Periodic Report page appears:
CanIt-Domain-PRO — AppRiver, LLC 182 CHAPTER 11. PERIODIC REPORTS
Figure 11.2: Add Periodic Report
To create the report:
1. Pick a name for the report and enter it in the appropriate box.
2. Pick a time when the report should be sent. You can pick daily, weekly or monthly reports. You can also select “On-Demand Only”. Such reports are never sent automatically, but are only generated on demand.
3. Enter an e-mail address to which the report should be sent. You can enter multiple addresses by separating them with commas.
4. Select a page size for the report (A4 or US Letter).
5. Pick one or more charts for the report by enabling the appropriate Add checkboxes.
6. Click one of the Submit Changes buttons.
11.4 Editing Periodic Reports
To edit an existing periodic report, click on the report’s name in the Name column. You alter the reports parameters, add or remove charts, or move existing charts up or down from the report editing page. To delete a periodic report, enable the appropriate Delete... checkbox and click Submit Changes.
CanIt-Domain-PRO — AppRiver, LLC 11.5. RUNNING A REPORT ON DEMAND 183
11.5 Running a Report on Demand
To run a specific periodic report on demand, enable the appropriate Run Now... checkbox and click Submit Changes. The report will be queued for processing. Note that it can take anywhere from a few minutes to a few hours for the report queue to be processed, so the report might take a while to be mailed out.
CanIt-Domain-PRO — AppRiver, LLC 184 CHAPTER 11. PERIODIC REPORTS
CanIt-Domain-PRO — AppRiver, LLC Chapter 12
Locked Addresses
12.1 Introduction to Locked Addresses
Locked Addresses are designed to solve the following problem: You want to give out your e-mail address to someone, but you don’t trust that person or organization not to turn around and give or sell it to others. You want an address that can only be used by the person or organization you give it to, and not by anyone else. CanIt-Domain-PRO has a complete solution to this problem. However, it does require some adminis- trative overhead before users can take advantage of the feature.
12.2 Preparing to use Locked Addresses
Before end-users can use locked addresses, you need to perform the following steps.
12.2.1 Create a new domain
Choose a new domain, specifically for locked addresses. This domain should be a subdomain of your “real” domain. For example, if you own the domain roaringpenguin.com, you might choose to place all your locked addresses in la.roaringpenguin.com. The domain you use for locked addresses should contain only locked addresses and should not be used for any “real” e-mail addresses.
12.2.2 Configure mail for the new domain
The next step is to configure the CanIt-Domain-PRO machine to receive mail for the new domain. Obviously, the first thing you need to do is publish an MX record for the domain. For example, if your locked address domain is la.roaringpenguin.com and your CanIt-Domain-PRO server’s name is canit.roaringpenguin.com, you might add a DNS record that looks like this:
la.roaringpenguin.com. 1d IN MX 1 canit.roaringpenguin.com.
CanIt-Domain-PRO — AppRiver, LLC 185 186 CHAPTER 12. LOCKED ADDRESSES
Also, you need to configure the CanIt-Domain-PRO machine to accept and discard all mail for the locked domain. (Mail should never be delivered to addresses in the locked domain, but just in case, there should be a mechanism to discard them.) Configuring Sendmail to accept mail for the locked domain is easy: Just add an entry in the access database. In our example, it would be:
To:la.roaringpenguin.com RELAY
(If you are running a CanIt-Domain-PRO Appliance, you can use Domain Routing from the Web interface instead of manually editing Sendmail configuration files.) The easiest way to configure Sendmail to discard mail for the locked domain is to make use of the virtusertable feature. Add an entry like this in virtusertable:
@la.roaringpenguin.com [email protected] and ensure that mail to [email protected] gets discarded (by making an alias from devnull to /dev/null.) Of course, you need to substitute your own locked address domain for la.roaringpenguin.com and your own CanIt-Domain-PRO server name for canit.roaringpenguin.com.)
12.2.3 Inform CanIt-Domain-PRO about the locked address domain
CanIt-Domain-PRO needs to know the domain you’re using for locked addresses, so it can treat any such addresses specially. In the Web interface, click on Administration : Global Settings and enter the locked address domain into the global setting G-10000 “Domain for Locked Addresses”
12.2.4 Associate each login name with an e-mail address
CanIt-Domain-PRO can only generate locked addresses if it has a real e-mail address for each logged- in user. For users in CanIt-Domain-PRO’s built-in user table (Section 6.5 on page 116), simply ensure that you enter an e-mail address for each user. For users authenticated via external means, the User Lookup method must return the user’s e-mail address upon login. For some User Lookup methods such as POP3 or IMAP that cannot return the e-mail address, you need to create an account-info script (Section 7.2.5 on page 157) and ensure that a mail=email-address attribute is always emitted for each login that should be permitted to use locked addresses. Once all of these steps in Sections 12.2.1 through 12.2.4 have been performed, the Locked Address feature is ready to use. Please consult the CanIt-Domain-PRO User’s Guide for details about how to use a Locked Address.
CanIt-Domain-PRO — AppRiver, LLC Chapter 13
Attachment Handling
CanIt-Domain-PRO can handle file attachments in a number of different ways. Messages can be delayed, rejected or held based on the attachment’s type. They can be scanned for viruses and held or rejected using one or more configured virus scanners. If desired, attachments can also be removed from the message and discarded, or held for access via a web-based system.
13.1 General Filename and MIME Type Rules
Whole messages can be rejected or held on a per-stream basis using the Filename Extensions or MIME Types rules. See the section entitled Blocklists, Allow Lists and Rules in the CanIt-Domain- PRO Users Guide for full details.
13.2 Delaying Attachments
On a realm-wide basis, it is sometimes useful to delay certain attachment types temporarily, without placing them in a stream’s quarantine. By delaying these attachments for a short period of time, you can give your virus scanners and RBLs time to catch up with new virus and spam content.
13.2.1 Configuring the Time Delay
Note: This section describes features that only the CanIt-Domain-PRO System Administrator can use. Configure how long CanIt-Domain-PRO should hold attachments by modifying Time in hours to delay messages with Delayed Attachments under Global Settings.
13.2.2 Creating Delay Rules
To create a delay rule, click on Administration and then Delayed Attachments. The Delayed At- tachments screen appears:
CanIt-Domain-PRO — AppRiver, LLC 187 188 CHAPTER 13. ATTACHMENT HANDLING
Figure 13.1: Delayed Attachments
To add a rule:
1. Enter a filename pattern in the Filename Pattern box. A filename pattern is normally inter- preted as a filename extension. For example, exe will match a file with the extension .exe. Note that the pattern should not contain a period. If a filename pattern begins with ˆ, then it matches an entire filename. For example, the pattern ˆbad.exe matches (only) the filename bad.exe. 2. Enter a comment in the Comment box. This will help you remember why you are delaying the given filename pattern 3. Click Submit Changes to add the rule.
Note: Attachment-delaying is global. It cannot be adjusted on a per-stream basis.
13.2.3 How It Works
As an administrator, you may configure any number of file extensions or full filenames to be delayed. When a message arrives matching that filename or extension, it will be held in a special @@DELAYED stream for the number of hours specified in the Time in hours to delay messages with Delayed Attachments configuration. Once that time has elapsed, the message is automatically released from the @@DELAYED quarantine, proceeding through the CanIt-Domain-PRO filtering process where normal scanning will proceed as if that mail had just arrived. Should it be necessary for a message to be released from @@DELAYED early, the admin user (or other user with appropriate permissions) may manually release it. Note, however, that a message released from @@DELAYED may be re-quarantined in its normal stream because of spam-scoring rules. That is because messages released from @@DELAYED are scanned by CanIt-Domain-PRO as if they had never been seen before; CanIt-Domain-PRO does not correlate what it believes to be a brand new message with anything in the @@DELAYED stream.
CanIt-Domain-PRO — AppRiver, LLC 13.3. STRIPPING ATTACHMENTS 189
13.3 Stripping Attachments
In addition to delaying, holding or rejecting mail based on characteristics of attachments, CanIt- Domain-PRO can strip attachments out of messages before forwarding the message. You can con- figure CanIt-Domain-PRO to strip out attachments and store them for retrieval via the Web interface, or simply to strip them out and discard them. Attachment-stripping rules can be set per-stream, but only the realm administrator can create or edit attachment-stripping rules; normal users cannot. To create attachment-stripping rules:
1. Click on Rules and then Attachment Stripping. You see the Attachment Stripping Screen:
Figure 13.2: Attachment-Stripping Rules
2. Enter a filename pattern in the Filename Pattern box. This pattern is interpreted exactly as for Delayed Attachments.
3. Enter a comment in the Comment box.
4. Choose an Action setting to determine how CanIt-Domain-PRO handles the filename pattern:
• Keep in Message indicates that CanIt-Domain-PRO should not strip the attachment out. This setting can be used in a particular stream to override settings in default. • Strip and Store on Server indicates that CanIt-Domain-PRO should remove the attach- ment and store it in the PostgreSQL database. CanIt-Domain-PRO will also add a message indicating that the attachment was stripped, and provide a link whereby the message re- cipient can retrieve the attachment. • Strip and Discard indicates that CanIt-Domain-PRO should remove and discard the at- tachment. CanIt-Domain-PRO will add a note to the message indicating that the attach- ment was discarded and cannot be retrieved.
5. If you chose Strip and Store on Server as the Action, then enabling the Require Approval? checkbox will force administrators to approve the release of held attachments.
6. Click Submit Changes to create the rule.
CanIt-Domain-PRO — AppRiver, LLC 190 CHAPTER 13. ATTACHMENT HANDLING
13.3.1 Approving the Release of Stripped Attachments
If an attachment rule specifies Require Approval, then when an end-user clicks on the link to retrieve the attachment, he or she will receive a notification stating that an administrator must approve the release of the attachment, as well as a code to supply to the administrator. To approve the release of an attachment:
1. Click on Rules : Attachment Stripping
2. Click on the Approve Attachment for Release link near the bottom of the page.
3. Enter the code supplied to you by the administrator.
4. Preview the attachment if necessary.
5. Click on Approve for Retrieval if you wish to allow the end-user to download the attachment.
CanIt-Domain-PRO — AppRiver, LLC Chapter 14
URL Proxying
CanIt-Domain-PRO’s URL Proxying feature can help mitigate phishing attacks that trick users into visiting hostile web sites and entering sensitive information. It does this by rewriting URLs in message bodies to go to a proxy page that warns users not to enter sensitive information. Users can then click on a link in the proxy page to visit the original URL. We call the rewriting of the link redirecting the link. Here is a screenshot showing what happens when a user clicks on a redirected link:
Figure 14.1: Redirected Link
In Figure 14.1, the original sender sent an email containing the link http://www.cnn.com/WORLD/?hpt=sitenav. CanIt-Domain-PRO redirected the link to its proxy page. The proxy page shows the user the original link, the server hostname, and (if it can be determined) the approximate location of the server. It also displays a warning not to enter sensitive information. This can help to educate users about the legitimacy of the site and to remind them not to enter sensitive information. To continue to the original site, the user merely needs to click on “I understand and wish to follow the link.”
CanIt-Domain-PRO — AppRiver, LLC 191 192 CHAPTER 14. URL PROXYING
14.1 Configuring URL Proxying
By default, CanIt-Domain-PRO proxies only URLs on the Known Phishing URLs list (Section 6.4.2). To enable URL proxying for other URLs, you need to create URL proxying rules. There are two basic strategies for using the URL proxying feature:
1. Enter a list of safe domains that should not be redirected, and have CanIt-Domain-PRO redirect everything else. This is the safest approach, but can be annoying as most URLs end up getting redirected.
2. Enter a list of questionable domains that should be redirected, and do not redirect anything else. If you have a list of commonly-abused domains such as free form-creation sites, this can be a viable method of cutting down on phishing while keeping the annoyance factor to a minimum.
To create URL proxying rules, click on Rules and then URL Proxy. The URL Proxy Rules page appears:
Figure 14.2: URL Proxy Rules
• To enable URL proxying, set “Enable URL Proxy?” to Yes. This is a normal stream setting, so if you set it in the default stream, it is inherited by other streams in the current realm and all subrealms.
Note: If the URL proxy is enabled, then CanIt-Domain-PRO proxies URLs for inbound mail and deproxies them (ie, undoes the wrapping) for outbound mail. An outbound message is defined as one that was forced into a stream by virtue of a Known Networks entry; an inbound message is one that was not.
CanIt-Domain-PRO — AppRiver, LLC 14.2. PROXYING KNOWN PHISHING URLS 193
Scanning messages for URLs and replacing them may be expensive, so if a stream does not need URL proxying, it is best to leave the setting at No. If the setting is No, then any URL Proxy Rules are ignored and the Known Phishing URLs list is not used.
• Set your default policy by entering a domain of * and either enabling or disabling the Redirect checkbox. In Figure 14.2, we proxy URLs by default. The possible policies are:
– Never — do not proxy URLs within this domain. – Always — always proxy URLs within this domain. – If Tagged as Spam — proxy URLs within this domain only if the email is tagged as spam and the stream is in tag-only mode. – If Domain is Newly-Seen — proxy URLs if the domain name of the URL host is newly seen (see Section 6.13.6 on page 133). Note that this selection makes little sense for a par- ticular domain name like paypal.com or google.com, but is useful for the wildcard domain * or for top-level domains such as info or biz.
• Set policies for specific domains by entering them in the Domain box and enabling or dis- abling the Redirect checkbox. Note that a rule for a domain like example.com applies to example.com and all subdomains unless there is a more specific rule. URL proxy rules follow the normal stream inheritance. CanIt-Domain-PRO uses the first match- ing rule in the most-specific stream to determine whether or not to proxy a URL.
• You can delete a URL Proxy Rule by enabling the appropriate checkbox in the Delete? column.
• Click Submit Changes to apply your changes.
14.2 Proxying Known Phishing URLs
CanIt-Domain-PRO maintains an updated list of URLs believed to have been used in a phishing at- tempt. If one of those URLs is encountered in a stream where URL proxying is enabled, the URL is always proxied regardless of any domain rules. In addition, if a user clicks on the modified link, he or she is not given an option to visit the original URL. Instead, CanIt-Domain-PRO displays a message indicating that the original link led to a suspected phishing site.
14.2.1 Known Phishing Test Point
The nonexistent URL canit-url-proxy-testpoint.example.com may be used to test the URL proxy. If you send yourself an email containing the text http://canit-url-proxy-testpoint.example.com, then CanIt-Domain-PRO should treat it as a known phishing URL.
CanIt-Domain-PRO — AppRiver, LLC 194 CHAPTER 14. URL PROXYING
CanIt-Domain-PRO — AppRiver, LLC Chapter 15
SMTP Server Testing
CanIt-Domain-PRO permits you to run a test SMTP session against a back-end SMTP server. It displays the complete SMTP session and this lets you diagnose problems that may exist.
15.1 An SMTP Primer
Internet email is delivered using a protocol called the Simple Mail Transfer Protocol, or SMTP. SMTP runs over TCP, usually on port 25. In an SMTP session, there are two computers involved. The machine attempting to send email is the one that initiates the connection, and it is called the SMTP Client. The machine that is intended to receive the email accepts an incoming connection from the client and is called the SMTP Server. The data exchanged between an SMTP client and an SMTP server is human-readable plain text. It consists of a number of client commands, each of which is responded to with a server reply. The only exception is that immediately upon the client connecting to the server, the SMTP server issues a server reply called the server banner, without waiting for a command from the client. The flow of an SMTP session is shown in Figure 15.1:
CanIt-Domain-PRO — AppRiver, LLC 195 196 CHAPTER 15. SMTP SERVER TESTING
1. Client connects SMTP Client 2. Server sends banner SMTP Server
3. Client sends command
SMTP Client 4. Server sends reply SMTP Server
5. Client sends command
SMTP Client 6. Server sends reply SMTP Server
n−1. Client sends QUIT
SMTP Client n. Server closes connection SMTP Server
Figure 15.1: SMTP Session
Each server reply consists of a three-digit reply code followed by additional text. The first digit of the reply code indicates the success or failure of the preceding client command; the first-digit responses are as follows:
• 2 indicates a successful reply. It means that the preceding client command succeeded and the server is waiting for the next command.
• 3 indicates a provisionally successful reply. It means that the preceding client command suc- ceeded, but more information is needed before an overall success or failure status can be re- turned. This reply code is not frequently used and will not appear in CanIt-Domain-PRO’s SMTP tester.
• 4 indicates a temporarily unsuccessful reply, often called a tempfail. It means that the preceding client command failed, but that it may succeed at some point in the future if the client retries the SMTP session. Examples of conditions that could elicit such a response are full disks or problems reaching a directory server.
• 5 indicates a permanently unsuccessful reply, often called a permfail. It means that the preceding client command failed and that there is no point in the client retrying later on because it is not
CanIt-Domain-PRO — AppRiver, LLC 15.2. TESTING AN SMTP SERVER 197
likely to ever succeed. A condition that could elicit such a response is an attempt to send mail to a nonexistent recipient.
15.2 Testing an SMTP Server
The SMTP server-testing feature can be accessed from three places in the Web interface:
• From the Test link next to each Verification Server entry.
• From the Test link next to each Domain Routing entry.
• From the Test links on the Domain Overview page.
When you access the SMTP server-testing feature, the SMTP Server Test Parameters page appears:
Figure 15.2: SMTP Server Test Parameters
To run the test, enter the following parameters:
• Enter the name of the domain to test in the domain name box. Note that this may already be filled in for you.
• Provide the first part of a valid email address. For example, if you are testing the domain example.com and you know that [email protected] is a valid email address, enter info in the second box.
• Optionally enter the server name and IP in the next box. If you leave this box blank, the server will be taken from the Verification Server or Domain Routing entry. Note: Only the site admin- istrator can test arbitrary servers. Realm administrators can only test servers that are Verification Server or Domain Routing entries.
• If a domain has both a Verification Server and a Domain Routing entry, select which server to test. This choice appears only for domains that do in fact have both types of entries.
• Click Run the Test to test the SMTP server.
CanIt-Domain-PRO — AppRiver, LLC 198 CHAPTER 15. SMTP SERVER TESTING
15.3 SMTP Test Results
Once you run a test, the Test Results page appears:
Figure 15.3: SMTP Server Test Results
The results are displayed in a three-column table. The columns are:
• Time (s) is the time in seconds that has elapsed since the initial SMTP connection was made.
• Source is the source of the message. It is one of Info, meaning an informational message and not part of the SMTP session; Server, meaning a server reply, or Client, meaning a client command.
• Message is the specific message, reply or command. The server reply codes are highlighted; client commands are shown in bold.
CanIt-Domain-PRO — AppRiver, LLC 15.3. SMTP TEST RESULTS 199
Let’s step through the SMTP session in Figure 15.3:
1. At time 0.0, CanIt-Domain-PRO successfully connected to the SMTP server vanadium.roaringpenguin.com.
2. At time 0.099, the server replied with a successful response code 220 and its banner.
3. Next at time 0.099, the client sent its first command: EHLO colo3.roaringpenguin.com
4. At time 0.135, the server sent back a multi-line reply. Note that all but the last line have a dash instead of a space after the reply code. The multi-line reply has reply code 250, indicating that the EHLO command was successful.
5. Next at time 0.135, the client sent a MAIL From: command.
6. At time 0.167, the server responded to the MAIL From: command with a successful reply code.
7. Next at time 0.167, CanIt-Domain-PRO informed us that it was going to attempt to send mail to a valid email address.
8. And next at time 0.167, the client sent a RCPT To: command, specifying the email recipient.
9. At time 0.201, the server replied with the code 250, indicating that the preceding RCPT To: command was successful.
10. Next at time 0.201, CanIt-Domain-PRO informed us that it was going to attempt to send mail to a (likely) invalid address.
11. Finally at time 0.201, CanIt-Domain-PRO sent a RCPT To: command that specified a recipi- ent that is very unlikely to exist.
12. At time 0.233, the server replied with the code 550, which indicates a permanent failure. The preceding RCPT To: command failed and is not likely to succeed in future.
13. At time 0.234, the client sent a RSET command which throws away everything done so far in the SMTP session.
14. At time 0.265, the server indicated the success of the RSET command.
15. Next at time 0.265, the client sent a QUIT command.
16. Finally, at time 0.296, the server acknowledged the success of the QUIT command and closed the connection.
CanIt-Domain-PRO — AppRiver, LLC 200 CHAPTER 15. SMTP SERVER TESTING
CanIt-Domain-PRO — AppRiver, LLC Chapter 16
CanIt Storage Manager
16.1 Storage Manager Concepts
Normally, CanIt-Domain-PRO stores all incident-related data in the PostgreSQL database. For many sites, this works very well and there is no need for any alternate storage mechanisms. However, for large sites, storing large amounts of text in the database can be very burdensome, leading to very large databases and the consequent very long database dump and VACUUM processes. To alleviate this problem, CanIt-Domain-PRO ships with a program called canit-storage-manager. This program allows you to store large textual data in the file system rather than in the PostgreSQL database. The benefits of using the storage manager are:
1. The large amounts of text do not have to be dumped with each database backup, and they do not have to be VACUUMed.
2. Because the data are stored as ordinary files, you can easily back up and synchronize the data to other machines.
3. canit-storage-manager is optimized for the quick storage and retrieval of textual data, so it reduces the burden on the database server.
4. canit-storage-manager can be run on a different machine from the database server, which improves scalability.
CanIt-Domain-PRO — AppRiver, LLC 201 202 CHAPTER 16. CANIT STORAGE MANAGER
16.1.1 Principles of Operation
Figure 16.1 illustrates how the storage manager works:
ticker scanner
TCP traffic TCP traffic
TCP traffic canit−storage−manager Web UI Disk traffic
TCP traffic
File scanner System
Figure 16.1: CanIt Storage Manager
• The storage manager daemon runs on one machine and stores data locally on that machine’s file system.
• The scanners, ticker and Web interface processes (running on the same machine or in general on different machines) communicate with the storage manager daemon via a TCP connection.
• The scanners, ticker and Web interface make requests to fetch and store data and the storage manager daemon carries out those requests.
• Old data are expired by the cron job. The storage manager daemon supports a special “purge” request to delete old data.
CanIt-Domain-PRO — AppRiver, LLC 16.2. CONFIGURING THE STORAGE MANAGER 203
16.2 Configuring the Storage Manager
Before configuring the storage manager, you need to make the following decisions:
• You need to pick one or more machines to run the storage manager. These machines should be fast with plenty of memory and (most importantly) fast disks.
• You need to pick a directory under which the storage manager can store data. (It has to be the same directory on each machine that runs storage manager.) Be sure there is sufficient disk space for your expected mail storage! The required disk storage is given approximately by the following formula. (Note that this is a worst-case estimate. It assumes that 100% of your mail volume is spam and that every message is larger than 8kB and is held locally.)
S = (Dsig × M ×V) + (Ddata × 8kB ×V) + (Ddata × M ×V) where:
– S is the required amount of disk space. – V is the average number of messages received in a day. – M is the average size of a message.
– Dsig is the number of days before you expire old Bayes signatures.
– Ddata is the number of days before you expire old data. For example, if you receive 50,000 messages per day averaging 20kB per message, you retain Bayes signatures for 3 days and you expire old data after 28 days, the required disk space is: S = (3 × 20 × 50000) + (28 × 8 × 50000) + (28 × 20 × 50000) = 42200000kB or about 42GB.
16.2.1 Enabling the Storage Manager
Before using the storage manager, ensure that all machines in your CanIt-Domain-PRO cluster can connect to the storage manager daemon on port 6568 (or whatever port you choose for it to listen on.)
16.2.2 The Configuration Wizard
Once you have decided on the machine and directory, you can begin configuring the storage manager from the Web interface. Click on Setup and then Storage Manager Wizard.
1. First, you are asked whether or not you wish to use the storage manager. Answer Yes. Then click Next. The storage manager configuration page appears:
CanIt-Domain-PRO — AppRiver, LLC 204 CHAPTER 16. CANIT STORAGE MANAGER
Figure 16.2: Storage Manager Configuration
2. Enter the following information into the wizard:
(a) For each host in your cluster, select whether you want the host to run storage manager in Read/Write mode, Read-Only mode, or not at all. (Normally, you should never run storage manager in Read-Only mode; this mode is intended only when you are retiring a storage manager node and want to leave it in the pool until all data on it expires.) Note: If you change a Storage Manager node from Read/Write to Read-Only or vice-versa, you must run /etc/init.d/canit-system restart-gracefully on that node af- ter finishing the Storage Manager wizard. Otherwise, the change will not be picked up by the Storage Manager daemon. (b) If you have more than one host running a storage manager daemon and you want CanIt- Domain-PRO to store data only on some subset of them, enter the number of hosts on which to attempt writes in the “Number of Copies to Write” box. (c) If CanIt-Domain-PRO is writing more than one copy of the data and you want it to continue operating even if some writes fail, enter the number of writes required to succeed in the “Success Threshold” box. (d) Enter the port on which the storage manager daemon should listen. The default port is 6568. (The port must be the same for all storage manager hosts.) (e) If you want to restrict the daemon to listen on a particular IP address, enter it. Normally, you should leave this field blank. If you are running storage manager on more than one host, you must leave this field blank.
Once you have entered the settings, click Next.
3. Review the settings and then click Finish.
CanIt-Domain-PRO — AppRiver, LLC 16.2. CONFIGURING THE STORAGE MANAGER 205
16.2.3 Local Configuration
On each host, a number of settings in the [storagemanager] section of /usr/share/canit/ canit.conf control various aspects of the storage manager. If you want to change the settings, create a [storagemanager] section in /etc/mail/canit/canit.conf; do not edit /usr/ share/canit/canit.conf directly. The settings are: pidfile (string) A file used by the Storage Manager server to write its process ID and to lock against concurrent Storage Managers. The default value is /var/run/ canit-storage-manager.pid. rootdir (string) The root directory under which data are stored. The default value is /var/lib/ canit-storage-manager. user (string) The UNIX user as which the Storage Manager server should run. The default value is defang. client retry delay (integer) specifies the delay in reconnecting to a dead storage manager node. If a CanIt-Domain-PRO cluster node fails to connect to a storage manager node, it will not retry the connection for client retry delay seconds. This can help prevent a dead storage manager node from bogging down the clients in blocked connect calls. client connect timeout (integer) specifies the timeout in seconds for a connection attempt to a Stor- age Manager node. The default is 5 seconds. client operation timeout (integer) specifies the timeout in seconds for a read or write operation to a Storage Manager node once connection has been established. The default is 20 seconds. order (string) specifies the order in which to try Storage Manager nodes. The default is “auto”, in which case CanIt-Domain-PRO periodically measures the latency to each Storage Manager node and accesses them in order of increasing latency (fastest to slowest). If you want to specify a particular order, set the value to a space-separated list of fully-qualified host names. The hosts will be tried in the order given. If you do not specify all the hosts, then any remaining hosts are tried after the ones specified by the order parameter.
16.2.4 Starting the Storage Manager
Once the settings have been saved, you should log in to each host that will run the storage manager daemon. Become root and start the storage manager daemon: # /etc/init.d/canit-system check (Your canit-system program may be located in /usr/share/canit/scripts/canit-system instead.) The canit-system startup script should run on bootup; it will start the Storage Manager if required.
CanIt-Domain-PRO — AppRiver, LLC 206 CHAPTER 16. CANIT STORAGE MANAGER
16.2.5 Data Stored in the Storage Manager
Once the storage manager is enabled, CanIt-Domain-PRO stores the following data in it rather than in the PostgreSQL database:
• Bayes signatures.
• Message previews (the first portion of an incident’s message).
• Entire messages (if the message is being held locally for some reason.)
In addition, CanIt-Domain-PRO uses the storage manager rather than the database for collecting statis- tics. These statistics are periodically summarized out of storage manager and the summaries are placed in the database.
16.3 Backup Considerations
Once you start using the storage manager, the nightly database dump will not contain all of the infor- mation about incidents. In addition to backing up the nightly database dump, you should also back up the entire storage manager directory tree. (This directory is specified in /etc/mail/canit/ canit.conf as the rootdir setting in the storagemanager section. If there is no rootdir setting, then the default path is /var/lib/canit-storage-manager.) The files in that directory are ordinary files; you can back them up with tar or rsync or your favourite backup tool. However, there are many, many small files within many, many directories and subdirectories. Test to confirm that your backup tool can handle the directory. The best time to back up Storage Manager is after the nightly cron job has finished. This is because (a) expired data will have just been purged; and (b) the system should be less busy, resulting in less contention for disk I/O. If you have more than one CanIt-Domain-PRO server (in other words, a cluster) then it is best to run storage manager on multiple CanIt-Domain-PRO servers rather than using a backup tool. See section 16.4.
16.4 Running multiple Storage Managers
If you have more than one CanIt-Domain-PRO server running in a cluster then we strongly recommend running storage manager on at least two servers. There are several advantages:
• Storage Manager automatically load balances between its nodes;
• Multiple redundant copies of the data eliminate the need for backups;
• When migrating a server, you can skip the process of migrating storage manager as the other nodes will carry the data.
CanIt-Domain-PRO — AppRiver, LLC 16.5. PS OUTPUT 207
16.5 ps Output
If possible, canit-storage-manager changes the string shown by the ps command to reflect what it is doing. For example, ps might show the following output: canit-storage-manager: 10.0.0.1 scanner_6448 store bayes_sig 19819
The output above means that this instance of the storage manager is connected to the scanner with process-ID 6448 on the machine 10.0.0.1. It is currently executing the command “store bayes sig 19819”.
CanIt-Domain-PRO — AppRiver, LLC 208 CHAPTER 16. CANIT STORAGE MANAGER
CanIt-Domain-PRO — AppRiver, LLC Chapter 17
Searching Logs
17.1 Introduction
CanIt-Domain-PRO has the ability to index mail logs in the PostgreSQL database and search them. This can be used to diagnose many mail problems such as missing messages, duplicate messages, etc.
Note: The log-searching feature is available only on our Debian-based appliance build. It is not available in the source or RPM versions of CanIt-Domain-PRO. See the CanIt-Domain-PRO Installation Guide for details on installing the log-searcher. In addition to presenting search results from the log files, CanIt-Domain-PRO also annotates the log- lines to provide a clear explanation of what each line means. This can greatly ease troubleshooting.
17.2 Log Basics
CanIt-Domain-PRO uses the Sendmail program to transfer mail. It also uses the MIMEDefang filter- ing tool as the basis for its filtering. There are therefore three sources of log lines:
1. Sendmail.
2. The core MIMEDefang tool.
3. CanIt-Domain-PRO itself.
The log indexer groups log lines for a given message into a log document. A log document consists of the set of log lines that describe the process of one message transmission through the CanIt-Domain- PRO system. The common element between different log lines that allows them to be grouped together is the Send- mail queue ID. This is an identifier assigned by Sendmail to each message transmission. A typical queue ID might look like this: oBGIkIUj026238
CanIt-Domain-PRO — AppRiver, LLC 209 210 CHAPTER 17. SEARCHING LOGS
17.3 Searching the Logs
There is a 30-minute delay between a log-line being created and the indexer indexing it. Therefore, you can search for log lines starting as far back as your logs go up until 30 minutes before the current time.
17.3.1 Performing a Search
Note: Only the system administrator or realm administrators can use the log-searching facility. In addition, the user must have permission to see quarantine contents. To search the logs, click on Administration : Search Logs. The Log Search page appears:
Figure 17.1: Log Search Page
The Log Search page lets you build up a complex search query and then execute it. Here’s how log-search queries work:
• Start date and End date restrict the time interval over which the search is performed. These
CanIt-Domain-PRO — AppRiver, LLC 17.3. SEARCHING THE LOGS 211
are not actually part of the query.
• A query is a list of zero or more groups. Each group is evaluated as a unit before evaluating the next group.
• Each group consists of one or more expressions. Each expression is evaluated as a unit.
• An expression consists of a field, a relation and some data. These will all be explained soon.
• Within a group, expressions are joined with AND, OR, AND NOT or OR NOT. The AND operator is evaluated with higher precedence than OR. (If you include NOT, the NOT negates the next expression.) Thus, for example, a query like:
(X = 1) AND (Y = 2) OR (A = 3) AND NOT (B = 4)
is evaluated as:
((X = 1) AND (Y = 2)) OR ((A = 3) AND (NOT (B = 4)))
• Within a query, groups are joined with AND, OR, AND NOT or OR NOT. Again, the AND operators have higher precedence than OR.
17.3.2 Fields
The possible fields for searching logs are:
• Incident ID lets you search for a specific CanIt-Domain-PRO incident ID.
• Queue ID lets you search for a specific Sendmail Queue ID.
• Sender lets you specify an envelope sender’s email address.
• From: Header Address lets you specify the email address appearing in a message’s From: header.
• List of Rules Hit lets you search the tests=xxx field of CanIt-Domain-PRO’s what= log line. The most useful way to use this field is with the contains relation. If you use that relation, you can search various rule types as follows:
– To search for a SpamAssassin rule such as HTML MESSAGE, enter the rule identifier ex- actly as shown. – To search for a Custom Rule with ID N, search for: ;CN( where the semicolon, the C and the ( are literal. – To search for an SPF result of xxx, search for: SPF(xxx: – To search for an DKIM result of xxx, search for: DKIM(xxx: – To search for a DMARC result of DMARC POLICY xxx, search for: DMARC(DMARC POLICY xxx).
CanIt-Domain-PRO — AppRiver, LLC 212 CHAPTER 17. SEARCHING LOGS
You can also search for filename extensions using List of Rules Hit. For example, to search for a docx extension, use: List Of Rules Hit Contains ext:docx To look for a zip-within-a-zip, use: List Of Rules Hit Contains ext:>zip
• Recipient lets you specify an envelope recipient’s email address.
• Source Relay IP lets you restrict results to messages relayed from a specific IP address.
• Destination Relay IP lets you restrict results to messages relayed to a specific IP address.
• Subject lets you specify the subject of a message.
• Message ID lets you specify a Message ID (found in the Message-Id: header of an email.)
• Reporting Host lets you restrict the search to messages processed by a particular host. Note that you need to specify the host name as it appears in the log file.
• Classification lets you restrict messages based on their classification. Possible values for clas- sification are:
– Accepted – Rejected – Discarded – Greylisted – Pending – Tagged – Streamed
• Stream lets you restrict results to messages within a given stream.
• Score restricts the results based on score.
• Reason restricts results to messages with the given reason=xyz entry in their logs.
• Detail restricts results to messages with the given detail=xyz entry in their logs. For more information about the reason and detail fields, see AppendixH, “Logging”.
• Realm lets you restrict messages to a particular realm. The Realm field is displayed only if you have access to more than one realm.
CanIt-Domain-PRO — AppRiver, LLC 17.4. SAVING LOG SEARCHES 213
17.3.3 Creating a Log Search Query
To create a log search query:
• Starting with a blank query, select a field and relation for the search. Enter the data to search for and click Add • Continue to refine the query by selecting additional fields and relations and entering additional data. Also, select one of AND, OR, AND NOT or OR NOT as the logical operator to join the new expression to the existing query. • Click Add to add the new expression to the current group, or click Add as New Group to start a new group. • Click Delete to delete the most recently-added expression if you made a mistake.
17.4 Saving Log Searches
CanIt-Domain-PRO permits you to save a log search and call it up later to redo the search. To save a log search:
1. Create the log query in the normal manner. 2. Enter the name under which you would like to save the search in the box to the right of the Save Search As... button. 3. Click Save Search As...
17.4.1 Managing Saved Log Searches
To manage saved log searches, click on Managed Saved Searches. The Saved Log Searches page appears:
Figure 17.2: Saved Log Searches
To recall a saved log search, click on the name of the search. The log-search page will appear with the query loaded from the saved search.
CanIt-Domain-PRO — AppRiver, LLC 214 CHAPTER 17. SEARCHING LOGS
To add a comment to a saved log search, enter the comment in the appropriate box and click Submit Changes. To delete saved log searches, enable the appropriate checkboxes in the Delete? column and click Submit Changes.
17.5 Log Search Results
After you click Add and Search to submit a log search request, CanIt-Domain-PRO returns a list of matching results. This list might look something like Figure 17.3:
Figure 17.3: Log Search Results
Within the results page:
• Click on the small up- or down-arrows next to each column to sort by that column in ascending or descending order. The current sort order is shown by the red arrow.
• Click on a Queue ID link to view the detailed log lines for that queue ID.
• If there is an incident associated with the logs, the message subject will be a link. Click on it to see the Incident Details page.
Note: Sometimes a group of log lines does not contain complete details about a message. In this case, CanIt-Domain-PRO acts as follows:
• If the subject could not be determined, CanIt-Domain-PRO displays the subject as (Not Logged).
• If the stream could not be determined, CanIt-Domain-PRO assumes the default stream.
• If the realm could not be determined, CanIt-Domain-PRO assumes the base realm.
It is important to remember that for queue retries and other fragmentary groups of logs, the subject, realm and stream may not be able to be determined.
CanIt-Domain-PRO — AppRiver, LLC 17.5. LOG SEARCH RESULTS 215
17.5.1 Detailed Results
If you click on a queue ID, the Detailed Results page appears:
Figure 17.4: Log Search Details
This shows each log line related to the message transmission. To see the timestamp in a more readable format, hover the mouse cursor over the timestamp. For a detailed explanation of a log line, click on the question-mark icon next to the line. You can expose details for all log lines by clicking Show All Explanations. Finally, if you need the raw log lines (for example, to send to someone for analysis), click on Show Raw Logs.
17.5.2 Downloading Log Lines
At the bottom of the log results page, you will see one or two links:
• Bookmarkable Link is a link that you can copy and paste or send via email to redo the currently-displayed log search.
• Download Logs is a link that permits you to download all logs that correspond to a particular query. The downloaded logs are in plain-text format that can be opened with a text editor.
CanIt-Domain-PRO does not always provide a Download Logs link. If the number of log search results is greater than the internally-configured MaxDownloadableLogs setting (default 100 log en- tries), then CanIt-Domain-PRO does not permit logs to be downloaded. The CanIt-Domain-PRO site administrator can increase the limit by creating a file under the CanIt- Domain-PRO web tree called site/config.d/99 logentries.php with the following con- tent:
CanIt-Domain-PRO — AppRiver, LLC 216 CHAPTER 17. SEARCHING LOGS
$Config['MaxDownloadableLogs'] = 500; ?>
In the previous example, the limit was raised from 100 to 500. When you download log lines, they are grouped by log host. Within a given log host, the lines are sorted chronologically. To sort all lines chronologically regardless of log host, use your text editor’s line-sorting feature or a utility similar to the UNIX sort command.
17.6 Forwarding Logs
CanIt-Domain-PRO has the ability to forward logs on a per-realm basis to other machines using the syslog protocol.
17.6.1 Enabling Log-Forwarding
By default, CanIt-Domain-PRO will not forward logs. To enable log-forwarding, the CanIt-Domain- PRO site administrator must edit the file /etc/mail/canit/canit.conf on each CanIt- Domain-PRO log host and add the following lines: [logindexer] forward logs = yes
17.6.2 Configuring Log-Forwarding
To configure log-forwarding, click on Administration : Forward Logs. The Log Forwarding Page appears:
Figure 17.5: Log Forwarding Page
Note: Only the CanIt-Domain-PRO site administrator can configure log-forwarding for arbitrary realms. If you are a realm administrator, the Log Forwarding Page allows you to configure log forwarding only for your current realm.
CanIt-Domain-PRO — AppRiver, LLC 17.6. FORWARDING LOGS 217
To forward logs for a particular realm:
1. Enter or select the realm name in the Realm column.
2. Type the IP address or host name of the destination host in the Log Host column. If you use UDP transport, you can enter multiple log hosts in a comma-separated list; in this case, log lines will be forwarded to each host. Additionally, you can use a different port for each host by following the host name or IP address with /port. If you use TCP transport, then you can only enter a single log host and cannot override the port.
3. Enter the port number in the Port column. The standard SYSLOG port is 514.
4. Select the transport (either UDP or TCP) from the Transport column.
5. Click Submit Changes
To disable forwarding for a realm, delete the entry with the Delete? check box, or enter a blank string for the host name.
Note: Forwarded logs are always forwarded with the mail facility and info priority, regardless of the original priority. Also, the entire original log line is forwarded including a high-resolution time-stamp. The receiving machine may log some redundant information with each received log line because of the way it is forwarded. Because CanIt-Domain-PRO must correlate log lines and ensure that all lines pertaining to a realm are forwarded (and no lines not pertaining to the realm are inappropriately forwarded), logs are not forwarded in real-time. There may be a delay of up to 30 minutes between a line being logged on the CanIt-Domain-PRO system and the line being forwarded to the remote host. Nevertheless, the original timestamp is preserved.
CanIt-Domain-PRO — AppRiver, LLC 218 CHAPTER 17. SEARCHING LOGS
CanIt-Domain-PRO — AppRiver, LLC Chapter 18
Tips
Managing spam requires constant attention, but there are many things you can do to reduce the work- load of the administrator. This chapter offers advice for fine-tuning CanIt-Domain-PRO and making it more effective.
18.1 Greylisting
Note: This section describes features that only the CanIt-Domain-PRO System Administrator can use. In the past, spammers would use open SMTP relays to send spam. With the advent of inexpensive residential broadband, many spammers use special software to send bulk mail directly from their PC’s. Because spammers want wide distribution, they want each message to be sent as cheaply as possible. Some spam software, therefore, ignores SMTP errors if a message cannot be delivered. CanIt-Domain-PRO can deal very effectively with software that never retries by sending a temporary failure indication at the end of DATA when mail from an unknown sender arrives. If you set the “Tempfail unknown senders on first transmission” stream setting to Yes, then CanIt-Domain-PRO uses the combination of sender e-mail address, recipient e-mail address, sending relay IP address and message subject to calculate a hash. If this hash has never been seen before, CanIt-Domain- PRO tempfails the message. Once the hash reappears, CanIt-Domain-PRO marks the host as “known to retry” and lets the message to proceed to content-scanning. A host marked “known to retry” is allowed to bypass greylisting for 40 days. There are some down-sides to using greylisting. Valid mail from new senders may be delayed by anywhere from 15 minutes to four hours, depending on the retry interval on the sending relay. You can avoid this delay by setting up a secondary MX record. In fact, you can simply give the CanIt- Domain-PRO machine a virtual interface with another IP address and publish this other IP address as a secondary MX record. In this way, when proper SMTP relays receive a temporary failure indication on the primary MX machine, they immediately try to send to the secondary MX machine. Often, spamware won’t retry. On a similar note, CanIt-Domain-PRO will not issue temporary failures for messages relayed from any server in a Known Network with Skip Greylisting configured (see Section 5.7 on page 65). If
CanIt-Domain-PRO — AppRiver, LLC 219 220 CHAPTER 18. TIPS
a message is received by such a server, greylisting will not be used. In some cases, this can cause greylisting statistics to be skewed. For example, if mail is initially received by a CanIt-Domain-PRO server and marked as greylisted, then is received by a secondary MX server and either relayed to the CanIt-Domain-PRO server, or to an internal mail server, the message will appear in the CanIt-Domain- PRO statistics as having been greylisted, even if it was received and processed. In general, we find that setting Tempfail unknown senders on first transmission to Yes is a cheap and effective way to reduce spam. WARNING: Some mailing list programs use “disposable” sender addresses which always change. These lists do not work well with greylisting. To work around the problem, you should always allow the domain of the mailing list sender. CanIt-Domain-PRO tries to detect disposable-address schemes. It ignores everything in the sender address following a plus sign or a dash followed by a digit. These rules catch most common methods for generating disposable addresses, but they are not exhaustive.
18.2 Don’t Trust Sender Addresses
Many spammers use one-time disposable sender addresses. Many addresses are not even valid. We do not recommend blocking addresses unless you receive many different spam messages from the same address. Therefore:
Blocking individual addresses is usually not effective. Always allowing known good addresses (for example, mailing-list sending addresses) can be very effective. The sender report may, however, highlight a persistent spam sender address which is worth blocking.
18.3 Don’t Trust Sender Domains
Just as sender addresses are often fake, sender domains are too. However, some domains are known spammers and these can be profitably blocked. The tip:
Blocking entire domains can be effective under limited circumstances. Always al- lowing domains is generally a bad idea because spammers often fake mail from good domains. Holding all mail from free e-mail services like Hotmail and Yahoo can be ef- fective if you use it in conjunction with always allowing of known good senders from those services. Use the domain report to help make these decisions.
18.4 You May Trust Relay Hosts
It is rather difficult to fake the IP address of the SMTP relay host, so this attribute can usually be trusted. We recommend using a DNS-based blocklist service in your Sendmail configuration file or the CanIt-Domain-PRO GUI to reject the most obvious offenders. However, if you receive multiple spam messages from a given relay host, it can be effective to block the host:
CanIt-Domain-PRO — AppRiver, LLC 18.5. CUSTOM RULES 221
Blocking a repeat-offender relay host is effective. Always allowing known good hosts such as internal hosts is effective and recommended. Use the host report to determine which hosts are persistent spam relays.
18.5 Custom Rules
18.5.1 General Recommendations
There are a few custom rules which are quite effective:
1. If you know that your CanIt-Domain-PRO server only accepts inbound mail from the Internet, then no server should ever claim to be in your domain in the HELO command. If your CanIt- Domain-PRO server is called canit.mydomain.tld, a custom rule to add 5 points if HELO ends with mydomain.tld can be very effective. In fact, you might want to make high-scoring rules which automatically reject messages with obviously-fake HELO arguments.
2. Similarly, no machine should ever put an IP address as the argument of HELO. Some spammers use random IP addresses here to confuse spam-reporting tools. A custom rule which “regexp- matches” HELO against ˆ\d+\.\d+\.\d+\.\d+$ can be quite effective.
3. Custom rules which specify Sender contains “offer”, “bounce”, “return” and “noresponse” can often trap spam. You should use only moderate scores on these rules, because some legitimate mail comes from such senders. However, adding a rule which scores around 3 for these patterns can help catch a lot of spam which might otherwise sneak under the scoring threshold.
4. Subject-matching rules for the most obnoxious spams are very effective. For example, Sub- ject regexp-match rules against v\Sagra and (increase|enlarge).*penis are very effective.
18.5.2 Things to avoid
Be very careful when writing custom rules, especially rules that can match on the message body. For example, a straightforward rule that contains “cum” in the body will match mail containing mail containing “document”, “cumulative”, “modicum” and at least 64 other common English words. Sim- ilarly, “sex” will match “sexton”, “Essex” and others. If you want to match words in a message body, we recommend that you use a regular-expression match, and use Perl’s word-boundary operators. For example, the Perl regular expression \bcum\b matches the word “cum”, but not “document”, “cumulative” or “modicum”.
18.6 Group High-Scoring Messages Together
We recommend that you set the default sort order to sort by Score, Descending. This groups high- scoring messages at the beginning and low-scoring messages at the end of the pending list. This makes it easier for the spam-control officer to dispose of the messages.
CanIt-Domain-PRO — AppRiver, LLC 222 CHAPTER 18. TIPS
18.7 AppRiver Best-Practices
At AppRiver, LLC, we’ve spent quite a bit of time analyzing spam and spammers. You may wish to try out some of our anti-spam rules to see if they work well for you. Here is a quick summary of the rules we use; they may inspire you to develop your own anti-spam rules.
• We use custom rules to add 4 to any message whose Sender contains “offer”, “noresponse”, “remove”, “marketing” or “promo”. These rules may be a touch aggressive for very busy sites, but are quite effective for smaller sites.
• Another custom rule adds 1.2 to any Relay containing “[” (left square bracket.) This indicates a reverse-DNS failure on the sending host, which is mildly correlated with spamming.
• We use a Spam threshold of 4.6, because we find the default of 5 is somewhat conservative.
• We use a discard threshold of 20; this seems quite safe.
• We set Tempfail unknown senders on first transmission to Yes. Again, this may be unaccept- able for some sites.
18.8 General Anti-Spam Tips
18.8.1 Use Receive-Only Addresses on your Web Site
Spammers love to extract e-mail addresses from Web sites, and not only do they use them for the obvious purpose of spam targeting, but also they use them as fake sender addresses. Therefore, we recommend a general policy of publishing only generic e-mail addresses on your Web site, like [email protected] and [email protected]. When you reply to inquiries, always use a real, personal e-mail address like [email protected]. This has two benefits:
1. If someone sends e-mail purporting to come from [email protected], you know immediately that it is spam, and you can reject it. You can block all your generic addresses inside CanIt-Domain-PRO.
2. If someone complains about receiving e-mail from one of the generic addresses, you can point to your policy and assure the recipient that the sender address was faked.
18.8.2 Do Not Reply to Spam
Do not ever reply to spam e-mail; such replies simply serve to validate your e-mail address. Similarly, do not visit Web sites purporting to offer opt-out services; they also serve to validate your address for further spamming.
CanIt-Domain-PRO — AppRiver, LLC Chapter 19
Security
Running a secure CanIt-Domain-PRO installation is relatively straightforward, but there are many issues you have to watch out for. This chapter gives you guidance on how to secure your CanIt- Domain-PRO installation.
19.1 Don’t Run as Root
The most basic security principle is to run as little software as root as possible. Therefore:
• Always create the Sendmail smmsp user and group, and do not run Sendmail suid-root. Instead, the permissions on the Sendmail executable should look like this: -r-xr-sr-x root smmsp sendmail That is, the sendmail binary should be owned by root, group smmsp and have mode 2555. • Always create the MIMEDefang defang user and group, and run MIMEDefang as defang. In /etc/mail/canit/canit.conf, enable mx user=defang in the [mimedefang] section.
19.2 Ownership and Permissions
All system configuration directories like /etc and their ancestors and descendants should be owned by root and writeable only by root. Here are suggested ownership and permissions for various files and directories. Note that where we use group root, your system may use wheel or some other group for root-owned files. File or Directory Owner Group Mode /etc/mail/canit and ancestors root root 0755 /etc/mail/canit/canit.conf apache defang 0640 /var/spool and ancestors root root 0755 /var/spool/MIMEDefang defang defang 0700 /var/spool/MD-Bayes defang defang 0755 /var/lib/canit-storage-manager defang defang 0700 The PHP files in Apache’s Web space root root 0644
CanIt-Domain-PRO — AppRiver, LLC 223 224 CHAPTER 19. SECURITY
19.3 SSH
The various nodes in a CanIt-Domain-PRO cluster communicate via SSH. Each node must be able to SSH to all other nodes on port 22. For intra-cluster communication to work, root SSH login must be permitted. However, you do not need to permit general root login because the CanIt-Domain-PRO nodes only use a forced command for communication. The safest setting in /etc/ssh/sshd config is therefore:
PermitRootLogin forced-commands-only
19.4 PostgreSQL Security
By default, PostgreSQL trusts any connection coming from the local host. Therefore, if you use PostgreSQL on your CanIt-Domain-PRO server with the default access rules, do not allow normal users to have shell accounts on the CanIt-Domain-PRO server. This cannot be emphasized strongly enough: If you allow normal users shell access on the CanIt-Domain-PRO server with PostgreSQL’s default setup, anyone can access or change the spam database. If you must allow shell accounts on the CanIt-Domain-PRO server, then you must password-protect your PostgreSQL installation. See the PostgreSQL documentation (“Authentication Methods” section) for details. You must also protect your database passwords:
• The file /etc/mail/canit/canit.conf must be owned by apache and group defang. Both the defang user and the apache user need read-access to these files, which should have mode 0640. (We assume your Web server runs as user apache; if not, substitute the Web server user as appropriate.)
For best security, we strongly recommend that you do not allow ordinary users to have shell accounts on your mail server. If the CanIt-Domain-PRO database server is on a different machine, you should not permit shell accounts on that machine either.
19.5 PHP Security
PHP has a parameter called register globals, which automatically sets global variables based on GET, PUT or COOKIE variables. This setting may be a security risk, and CanIt-Domain-PRO does not require it. We strongly recommend that you set register globals to off.
19.6 Network Security
When you log on to CanIt-Domain-PRO, your username and password are transmitted in cleartext. While you interact with CanIt-Domain-PRO, your browser passes a session cookie back so CanIt- Domain-PRO can keep track of your session. Both your password and the cookie are vulnerable to
CanIt-Domain-PRO — AppRiver, LLC 19.7. BACKUPS 225
network sniffing. If you interact with CanIt-Domain-PRO over an untrusted network, or a network whose traffic may be sniffed, you should use HTTPS and SSL encryption. Setting this up is beyond the scope of this manual, but CanIt-Domain-PRO should operate with no changes over HTTPS.
19.7 Backups
The daily CanIt-Domain-PRO cron job dumps a text backup of the spam database to the file /var/spool/Canit-Spam-DB-Backup/SPAM-DATABASE-BACKUP. You should back this file up regularly in case the CanIt-Domain-PRO server suffers a hardware or other problem. You should also make sure the file is not readable by normal users. You should also back up the entire directory tree rooted at /var/spool/MD-Bayes. If you are using the Storage Manager, you should also back up the Storage Manager directory on each Storage Manager node. Some CanIt-Domain-PRO settings are stored in /usr/share/canit as well as /etc/mail/canit; you should back up that directory any time that you change a file in it. You may wish to back up /etc/mail in its entirety to capture Sendmail configuration files in your backup as well. See Section E.3 for more information on automating backups to a remote location.
Note: When restoring from backups, never replace existing /etc/mail/ or /usr/share/canit files with backed up versions! Rather, use your backup versions as reference. Finally, please remember to back up any customizations you have made to your CanIt-Domain-PRO installation, including web interface files, custom account-info or other scripts, et cetera.
Note: When restoring from backups, be careful when replacing web interface files, especially (but not only) if you are restoring to a different version of CanIt-Domain-PRO than that from which your backup was made.
CanIt-Domain-PRO — AppRiver, LLC 226 CHAPTER 19. SECURITY
CanIt-Domain-PRO — AppRiver, LLC Appendix A
The Domain Configuration Wizard
A.1 Introduction
The Domain Configuration Wizard provides a simple way to quickly configure the most important settings for a domain. All of the pages in the Domain Configuration Wizard are available in greater detail in the Setup and Administration menus. However, because the Domain Configuration Wizard centralizes the important settings in one simple workflow, you may prefer to use it to set up new domains. To access the Domain Configuration Wizard, click on Setup and then Wizards. Click on Domain Configuration Wizard.
A.2 Entering the Domain Name
The first step in the Domain Configuration Wizard requires you to enter a domain name. (Figure A.1). Enter the domain name and click Next.
Figure A.1: Domain Configuration: Enter Domain Name
A.3 Picking a Realm
In the next page (Figure A.2), you are prompted to select a realm name. Enter the realm name and click Next.
CanIt-Domain-PRO — AppRiver, LLC 227 228 APPENDIX A. THE DOMAIN CONFIGURATION WIZARD
Figure A.2: Domain Configuration: Enter Realm Name
You may type the name of an existing realm, in which case CanIt-Domain-PRO maps the new domain into that realm. Or you may enter a new realm’s name, in which case the realm will be created and the domain will be mapped into that realm. If no data for the new domain exists yet, CanIt-Domain-PRO will suggest a realm name based on the domain name.
A.4 Configuring Streaming
The next step (Figure A.3) requires you to choose how mail for the domain should be streamed. Streaming is explained in detail in Chapter4.
Figure A.3: Domain Configuration: Configuring Streaming
You can configure streaming in several ways:
CanIt-Domain-PRO — AppRiver, LLC A.5. CONFIGURING AUTHENTICATION 229
• You can simply chop the domain part off the e-mail address so that mail for [email protected] goes into the stream user.
• You can chop the local part off the e-mail address so that mail for [email protected] goes into the stream example.net.
• You can keep the entire e-mail address as the stream name. This is the recommended method for most installations.
• You can invoke the User Lookup Wizard to set up a more complex streaming method (for example, using LDAP). The User Lookup Wizard is described in Chapter7.
Note that if you have created User Lookup methods (either in the past or after stepping through the User Lookup Wizard from the Domain Configuration Wizard), you will be presented with additional choices for streaming.
A.5 Configuring Authentication
Once streaming has been configured, you will be asked to configure authentication (Figure A.4).
Figure A.4: Domain Configuration: Configuring Authentication
CanIt-Domain-PRO — AppRiver, LLC 230 APPENDIX A. THE DOMAIN CONFIGURATION WIZARD
To allow end-users to log into CanIt-Domain-PRO and manage their quarantines, you can set up an authentication mechanism. From the Domain Configuration Wizard, you have several choices:
• IMAP allows you to authenticate users against an IMAP server.
• POP3 allows authentication against a POP3 server.
• Other allows you to skip setting up authentication. You can do it at a later time, or (if you do not want to allow end-users to log in) skip it entirely. You can also step through the User Lookup Wizard to set up a more complex authentication mechanism.
If you select IMAP or POP3, you will be prompted to enter the name (or IP address) of the IMAP or POP3 server. If CanIt-Domain-PRO should strip the domain name off the login name before attempt- ing to authenticate, set the “Strip domain name from login” parameter to Yes. You can also configure CanIt-Domain-PRO to validate SSL certificates and to use (or require) an encrypted connection to the POP3 or IMAP server. If you step through the User Lookup Wizard to create an authentication method, the newly-created method will be presented as an authentication choice when you return to the Domain Configuration Wizard.
A.6 Configuring Routing and Verification
Finally, CanIt-Domain-PRO will ask you to configure routing and verification (Figure A.5).
Figure A.5: Domain Configuration: Configuring Routing and Verification
CanIt-Domain-PRO — AppRiver, LLC A.7. SUMMARY 231
Note: Configuring routing via the Web interface is only available on CanIt-Domain-PRO appliance builds. If you are not running an appliance build, you will need to configure routing using Sendmail’s mail- ertable feature; consult the Sendmail documentation for details. To route mail for the domain, enter the host name or IP address of the back-end SMTP server that will accept e-mail for the domain. We strongly recommend configuring some method for CanIt-Domain-PRO to validate recipient ad- dresses. If you do not validate recipient addresses, CanIt-Domain-PRO is forced to accept mail for any address withing the domain, likely resulting in many failure notifications. If your back-end mail server validates recipients during the SMTP transaction, enter its name or IP address as the verification server. If it does not, you will have to leave the verification server blank and use some other method (such as LDAP streaming) to validate recipients.
A.7 Summary
After configuring routing and verification, CanIt-Domain-PRO will display a summary of your set- tings. Click Finish to make them take effect.
CanIt-Domain-PRO — AppRiver, LLC 232 APPENDIX A. THE DOMAIN CONFIGURATION WIZARD
CanIt-Domain-PRO — AppRiver, LLC Appendix B
Release Notes
Version 10.2.9 released on 2021-01-01
• BUG FIX: Added detection of invalid return data from cluster hosts when running cluster com- mands. This could cause a ticker task to die and not be cleared from the work journal. If the cause of the invalid return data is persistent, the task would continually fail and cause the rest of the work journal to never get executed.
• BUG FIX: Fixed a bug that would cause ”Reference found where even-sized list expected” messages in the logs.
• BUG FIX: DMARC verification did not fail on messages where the envelope sender was empty and no valid DKIM signature was present even when DMARC policy was ’reject’.
• DOCUMENTATION FIX: The fact that the CanIt API requires url-encoding of some characters that can appear in email addresses was documented with details for each affected character.
Version 10.2.8 released on 2020-10-01
• BUG FIX: Removed CheckFreshclamMirrors task since ClamAV no longer supports the check because Freshclam now uses a CDN rather than individual servers.
• BUG FIX: SPF query now specifies only version 1.
• BUG FIX: DMARC verification did not fail when DKIM was valid but identifiers were not in alignment.
Version 10.2.7 released on 2020-05-27
• UPDATE: ClamAV has been updated from 0.99.4 to 0.102.2.
CanIt-Domain-PRO — AppRiver, LLC 233 234 APPENDIX B. RELEASE NOTES
Version 10.2.6 released on 2020-01-16
• UPGRADES: New release to push out upstream updates for Appliance builds.
• BUG FIX: The log correlator did not handle timestamps extracted from from sendmail queue IDs starting in 2020 due to the encoded year value rolling over to zero.
Version 10.2.5 released on 2018-09-20
• UPDATE: SpamAssassin has been updated from 3.4.1 to 3.4.2.
• BUG FIX: The Shortener404 test could be DoS’d by a non-responsive URL shortener; DoS countermeasures have been added.
• BUG FIX: Autotask Integration will require an Integrator ID from Autotask as of 2018-12-31. This has been added to CanIt. (You don’t need to change anything in your Autotask settings.)
Version 10.2.4 released on 2018-09-11
• IMPROVEMENT: Exporting rules includes the last-hit date and hit-count for Sender, Domain and Network rules (assuming the log-indexer add-on is installed.)
• IMPROVEMENT (API): Pending Notification settings can be queried and set via the API.
• IMPROVEMENT (API): The GET /realm/@@/stream/@@/rules API call includes last-hit date and hit-count information where available.
• BUG FIX: The Domain Action page would not accept a domain with a leading period. This has been fixed.
• BUG FIX: The quarantine RSS feed feature would sometimes produce invalid RSS; this has been fixed.
• BUG FIX (Secure Messaging): The Reply-All function did not work correctly in some cases.
• BUG FIX: The new timezone support code could break the Web interface on ancient versions of PHP such as that shipped with CentOS 6.
Version 10.2.3 released on 2018-07-26
• BUG FIX: If the “Use Newly Seen Domain feature in URLProxy” feature is enabled (as it is by default), then malformed URLs in message bodies could cause CanIt to tempfail the message repeatedly. This has been fixed.
CanIt-Domain-PRO — AppRiver, LLC 235
Version 10.2.2 released on 2018-07-24
• NEW API CALL: you can obtain the actual message associated with an incident with GET /incident/
• MINOR NEW FEATURE: The URL proxy allows you to wrap a URL if the message is tagged or if the domain is new. Previously, you had to pick one or the other, but not both.
• MINOR NEW FEATURE (Secure Messaging only): In addition to being able to reply to the sender of a secure message, you can also now reply to all.
• MINOR NEW FEATURE (Secure Messaging only): If the Secure Messaging welcome mail template starts with “
• MINOR NEW FEATURE: You can view cached login credentials and delete specific cache entries.
• MINOR IMPROVEMENT: The “Show External Images” link is displayed both before and after messages in the message viewer.
• UPDATE: The Public Suffix List has been updated to the version as of 2018-04-16
• VALIDATION IMPROVEMENT: You cannot make network, sender or domain rules that are not valid networks, email addresses or domains, respectively.
• MINOR BUG FIX: Log search query form did not display proper time zone.
• BUG FIX: Our RPTN CA certificate had expired. We bundle an updated version.
• BUG FIX: Policy rules that replace recipients could misbehave if the recipient addresses were mixed-case; this has been fixed.
• BUG FIX: Performance of newly-seen domain lookups in URLs has been improved.
Version 10.2.1 released on 2018-04-10
• NEW FEATURE: You can decide whether or not to proxy a URL based on whether the domain name of the URL host is newly-seen.
• MINOR NEW FEATURE (Archiver): The importer can be told to include “deleted” items when importing a PST file.
• IMPROVEMENT: Don’t depend on podofopdfinfo any more; use qpdf instead.
• IMPROVEMENT: Custom Rules can usefully contain macros such as %{header from} in a regex match data field.
• POLICY CHANGE: Ignore void DNS lookups during SPF evaluation rather than erroring out if more than 2 void DNS lookups occur.
CanIt-Domain-PRO — AppRiver, LLC 236 APPENDIX B. RELEASE NOTES
• BUG FIX: Some minor time-zone related display bugs have been fixed.
• BUG FIX (Archiver): Archive search for recipients was case-sensitive; it is now case-insensitive as advertised in the manual.
• BUG FIX (Log Indexer): Log search for recipients was case-sensitive; it is now case-insensitive as advertised in the manual.
• BUG FIX: Internal calls to the alarm() system call could race; these race conditions have been fixed.
Version 10.2.0 released on 2018-02-13
• NEW FEATURE: Users can select their time zone (it is an inheritable stream setting) and all dates/times in the UI are expressed and accepted in the user’s time zone.
• NEW FEATURE: CanIt features experimental integration with Microsoft’s Azure Active Di- rectory (a hosted directory service.)
• IMPROVEMENT: Greatly improved and simplified the database failover code. It’s now easier to set up and more bulletproof. A lot of useless settings have been removed and the code has been cleaned up considerably.
• IMPROVEMENT: We use the “qpdf” helper program to analyze PDF documents rather than the older, slower and buggier “podofopdfinfo”
• IMPROVEMENT: In the case of multiple DKIM signatures, we evaluate them all and add up the corresponding scores.
• UPDATE: Update ClamAV to 0.99.3.
• BUG FIX: Improved detection of Microsoft Office documents with executable content inside them.
• BUG FIX (Archiver): Properly MIME-encode subjects when remailing archived messages.
Version 10.1.9 released on 2017-11-14
• SECURITY IMPROVEMENT: Our old 1024-bit DSA key for Roaring Penguin service access has been replaced with a 4096-bit RSA key.
• POLICY CHANGE (CanIt-Domain-PRO only): Auto-whitelist rules are *never* created in the base:default stream. Such rules would apply site-wide and we consider auto-creation of site- wide rules to be too dangerous.
• IMPROVEMENT: This version includes small improvements to the speed of custom rule eval- uation.
CanIt-Domain-PRO — AppRiver, LLC 237
• IMPROVEMENT: The OfficeMacro AutoOpen test also detects MS Office files that attempt to execute code without using macros. • CHANGE: If you mark a URL like that has only a hostname such as www.example.com/ as a phishing URL, then then *all* URLs that begin with www.example.com/ will be considered to be phishing URLs. • CHANGE (Log Search Only): A configuration setting in config.php allows you to limit the duration of log search queries to avoid having ridiculous searches launch very expensive SQL queries that run for a long time. • CHANGE (Archiver only): A search with no realm qualification searches all realms owned by a realm administrator rather than just the current realm. • BUG FIX: Release 10.1.8 broke the code that checks for verification server loops; this release fixes it again. In addition, CanIt now raises an anomaly if it detects a verification server loop. • BUG FIX: canit-setup-appliance on Debian Stretch would not detect any Ethernet interfaces. This is now fixed. • BUG FIX: Upon upgrade, the Roaring Penguin service SSH key would be enabled even if it had explicitly been disabled before. Now, when you run canit-service-key –disable, this fact is remembered and the key is never automatically re-enabled. • BUG FIX: The output of “lsar” changed format on Debian Stretch; we now handle both the old and new formats correctly.
Version 10.1.8 released on 2017-10-24
• IMPROVEMENT: CanIt will attempt to negotiate STARTTLS when it runs against a verification server. This only works on Debian Jessie or higher and only if the verification server advertises STARTTLS. • IMPROVEMENT (Archiver Only): In an Archive search, you can use the “is” relation with the “Subject” field. • IMPROVEMENT: Added support for PostgreSQL 10. At this point, the support for PostgreSQL 10 is considered experimental. • IMPROVEMENT (Log Searching): By default, log search queries for non-site-administrators are cancelled after 5 minutes. This limit is configurable in site/config.php; see con- fig.d/20 log search.php for details. • BUG FIX: The URL Proxy code would not rewrite URLs in a released incident even if the rules say that it should. This has been fixed. • BUG FIX: The default login filters for ActiveDirectory user-lookups now include (!(userAc- countControl:1.2.840.113556.1.4.803:=2)) in the login search filter to prevent disabled AD ac- counts from being able to log into CanIt. Existing user-lookups are NOT changed, so you will have to add this clause manually if you want to apply it to existing lookups.
CanIt-Domain-PRO — AppRiver, LLC 238 APPENDIX B. RELEASE NOTES
• BUG FIX: In certain rare cases, Custom, Archiver, etc. rules could compile to Perl code that did not evaluate as intended. This has been fixed. • BUG FIX: Implemented a workaround for a Linux kernel bug in the nightly cron job, which would sometimes complain about not being able to remove directories from a tmpfs file system because they were not empty. • BUG FIX: Fixed a typo in the CanIt::NewDomain module that could result in unnecessarily- high database load on very busy systems. • BUG FIX (Archiver Only): The search form defaults the relation for “Realm” and “Stream” fields to “is” rather than “contains”. • BUG FIX (Secure Messaging only): In some cases, activation codes could fail to work when setting up a new account. This has been fixed. • BUG FIX (Secure Messaging only): When running on a version of PHP older than 5.6, users experience authentication issues when trying to log in to the Secure Messaging portal. This bug was introduced in 10.1.7 and is fixed in this release.
Version 10.1.7 released on 2017-09-26
• NEW FEATURE: Roaring Penguin has a service that tracks when domains are first seen; this central database is now available for all CanIt installations and is used by the NewlySeenDomain test. • NEW FEATURE: The ability for users to see inherited rules is now controlled by a permission rather than always being available. • NEW FEATURE: A Custom Rule can match against the MX hosts of the envelope sender. This lets you detect parked domains, for example. • MAJOR IMPROVEMENT: The URL Proxy landing page follows redirects to show the ultimate destination of a link. It adds extra warnings if the link appears to point to an executable file or an archive. • IMPROVEMENT: If the greylist delay is increased due to a DNSBL hit, this fact is logged. • CHANGE: The “Welcome to CanIt” line shown on the login page has its own CSS class so it can be hidden by a theme customization. • CHANGE: The term “slave” with reference to scanning processes has been replaced by the term “worker”. • BUG FIX: A couple of places where canitd jobs could potentially hang have been fixed by imposing timeouts. • BUG FIX: If the auto-whitelist flag is set on a Known Network, auto-whitelisting is applied based on both the connecting server and the originating server in the cases when they are differ- ent.
CanIt-Domain-PRO — AppRiver, LLC 239
• BUG FIX: The various .pid files that CanIt creates are placed in /var/run rather than /var/spool/MIMEDefang. This tightens up the permissions.
• BUG FIXES (Secure Messaging): Incorrect template name for Secure Messaging was fixed. Minor database schema error was fixed; this does not affect functionality, only performance.
• BUG FIX (Secure Messaging): Secure Messaging now works correctly on Debian 9 (“Stretch”) and GnuPG version 2.
• BUG FIX (Secure Messaging): Replies via the web interface would not be encrypted under a rare edge-case condition. This has been fixed.
Version 10.1.6 released on 2017-08-15
• MAJOR NEW FEATURE: Almost all pages under the Rules top-level menu permit you to see inherited rules as well as the rules specifically in the stream you are viewing. You can toggle the inherited-rule view on or off. (Some rule types are considered sensitive and only the site administrator can view inherited rules in that situation.)
• NEW FEATURE: CanIt tracks all top-level domains from which mail has been received. 30 days after upgrading to 10.1.6, it will start adding one point for messages from “newly-seen” domains — that is, domains that first were seen 7 days or less ago. The score is adjustable under Rules > Plugins > NewlySeenDomain
• MAJOR IMPROVEMENT: For outbound mail (that is, mail forced into a stream by a Known Networks entry), CanIt will undo any URL-proxying that was done on the way in. This requires the URL Proxy feature to be enabled in the outbound stream.
• IMPROVEMENT: The GUI for Sender, Domain and Network Rules has been made consistent with other rule GUIs: A new blank row at the top of the page for a new rule, and checkboxes in a “Delete?” column for deleting rules.
• IMPROVEMENT: In /etc/mail/canit/failover.conf, you can specify NOT to supply the -z flag to rsync with the setting compress rsync=0. If you have a fast link between the primary and hot-standby database servers, it’s better not to compress.
• UPDATE: We now have packages for Debian 9 (“Stretch”) as well as Debian 9 ISO images. The Debian 9 packages and images are currently considered experimental.
• UPDATE: Several new country codes were added to Rules > Countries.
• UPDATE: CanIt’s public suffix list has been updated.
• MINOR FIX: All CanIt links referring to http://www.roaringpenguin.com have been changed to refer to https://www.roaringpenguin.com instead.
• MINOR FIX: If a Verification Server tempfails mail because of an unresolvable domain, we no longer raise an anomaly.
CanIt-Domain-PRO — AppRiver, LLC 240 APPENDIX B. RELEASE NOTES
• MINOR FIX: The nightly system checker handles domains with non-resolving NS records bet- ter than before; it raises a descriptive anomaly rather than issuing mysterious error messages to the logs.
• BUG FIX: The Custom Rule Compiler failed to handle regular expressions containing a single quote; this has been fixed.
• BUG FIX (Secure Messaging only): If a user account is disabled, CanIt would sometimes issue a misleading error message. This has been fixed.
Version 10.1.5 released on 2017-06-13
• POLICY CHANGE: Support for Debian Lenny (Debian 5.0) has been dropped. Debian Lenny has not had security support for 5 years and no-one should be using it any more.
• NEW FEATURE: Custom Rules, Delivery Policy Rules, and Secure Messaging rules can match against URLs in the message and also against just the URL hostnames.
• NEW FEATURE: The log-indexer lets you search for logs with attachment filename extensions by looking for “ext:XXX” in the List of Tests Hit field.
• NEW FEATURE: Most rules have their IDs hyperlinked to a pre-filled log search form that will find those rule hits in the logs.
• NEW FEATURE: The URL Proxy landing page lets a user vote that a URL is malicious.
• IMPROVEMENT: CanIt will de-proxy URLs in outbound message. Thus, if an incoming mes- sage had its URLs rewritten and it is forwarded out, the rewriting will be undone.
• IMPROVEMENT: The Verification Server page warns if a domain has an LDAP user-lookup (rendering the verification server superfluous.) A simlar warning appears on the Domain Map- ping page.
• IMPROVEMENT: