CommuniGate Pro
Version 5.2
E-mail
 
 
RPOP

RPOP Module

The CommuniGate Pro RPOP implements E-mail message retrieval using the POP3 Internet protocol (STD0053) via TCP/IP networks. While the POP module allows the CommuniGate Pro users to retrieve mail from their Server mailboxes, the RPOP module retrieves messages from other (remote) hosts and delivers them to user mailboxes or to other destinations.

For each registered user, the RPOP module can retrieve messages from several remote mailboxes. The RPOP module can retrieve mail for your entire domain using "Unified Domain-wide accounts" and distribute retrieved messages to their recipients.

The RPOP module supports non-standard MSN POP3 servers: if the remote host (server) name ends with .msn.com, the module uses the non-standard AUTH MSN method to log into that server.

Post Office Protocol (POP3) and Mail Retrieving

The RPOP module can be used when the CommuniGate Pro Server has a dial-up connection with dynamically assigned IP address, and thus the Server cannot receive mail via SMTP. The RPOP module polls the specified remote host (ISP) accounts, retrieves messages and stores them in the Server mailboxes.

Users with several mail accounts on several systems can instruct the RPOP module to poll those accounts, so all their mail is collected in their CommuniGate Pro Account.

The RPOP module supports Domain-Wide Accounts. A Domain-wide account is an account on the ISP or any other host that collects all messages sent to your domain. The RPOP module retrieves all messages from such an account and distributes them based on the addressing information in the message headers. The RPOP module can poll several Unified Domain-Wide Accounts.

The RPOP module activity can be limited using the TCP Activity Schedule. The module does not poll any remote account till the TCP Activity Schedule allows the Server to initiate outgoing network connections.

Configuring the RPOP module

Use the WebAdmin Interface to configure the RPOP module. Open the Mail pages in the Settings realm, then open the RPOP pages.

Processing
Log Level:   Processors:
Delay Failed Hosts for: Default IP Address:
Delay Failed Accounts for: Use Domain IP Addresses
Minimum Poll Period for Users: Use APOP
Maximum Number of Accounts per User: Allow Self-Poll
Log
Use the Log setting to specify what kind of information the RPOP module should put in the Server Log. Usually you should use the Major (message transfer reports) or Problems (message transfer and non-fatal errors) levels. But when you experience problems with the RPOP module, you may want to set the Log Level setting to Low-Level or All Info: in this case protocol-level or link-level details will be recorded in the System Log as well. When the problem is solved, set the Log Level setting to its regular value, otherwise your System Log files will grow in size very quickly.

The RPOP module records in the System Log are marked with the RPOP tag.

Processes
When you specify a non-zero value for this setting, the RPOP module starts to connect to the remote hosts and retrieve mail from accounts on those hosts. The setting is used to limit the number of simultaneous connections the RPOP module can initiate.
Use APOP
The RPOP can use the secure APOP authentication method when connecting to hosts that support this feature. If for any reason you want the RPOP module to always use the "clear text" passwords, disable the Use APOP option.
Default IP Address
This option selects the default source network address for POP3 connections. You can allow the server OS to select the proper address or your can explicitly select one of the server IP addresses as the default source network address.
Use Domain IP Addresses
This option selects source network addresses for POP3 connections made for Account-level RPOP records. If this option is selected, the RPOP module will use the first Assigned IP Address for the Domain the RPOP record belongs to.
If this option is not selected, or if the Domain does not have any Assigned IP Address, or if the RPOP record is a UDWA-record (see below), the RPOP module uses the Default IP Address as the source network address.
Delay Failed Hosts
When the RPOP module fails to connect to an external host, it marks the host as "failed" and stops polling all accounts on that host. The option specifies when the RPOP module should try to poll the failed host again.
Delay Failed Account
When the RPOP module fails to open a mailbox (wrong password, remote mailbox is locked, etc.), or if the connection fails when the module retrieves messages from a remote account, the module marks an account as "failed". The option specifies when the RPOP module should make the next attempt to poll the failed account.
Allow Self-Poll
Very often CommuniGate Pro users misunderstand the concept of remote account polling and specify their own CommuniGate Pro accounts as the "remote" accounts to be polled. This creates message loops and wastes Server resources. If this option is not selected, the RPOP module checks the network address of the remote POP server it has to connect to. If that address is one of the CommuniGate Pro Server own network addresses, the "remote" account is not polled.
Mimimum Poll Period for Users
If some users are allowed to specify their own individual RPOP accounts, they may select too short Poll Periods, generating a lot of network traffic and consuming the server resources. Use this option to set the minimum value the Server users can specify in their Poll Every remote account settings. This limit applies to users only. The administrator can always specify any Poll Period for the Unified Accounts and for individual RPOP accounts.
Maximum Number of Accounts per User
If some users are allowed to specify their own individual RPOP accounts, they may specify too many accounts, generating a lot of network traffic and consuming the server resources. Use this option to limit the number of RPOP accounts the Server users can specify. This limit applies to users only. The adiminitrator can always specify any number of Unified and individual RPOP accounts.

Click the Update button to modify the RPOP module settings.

Specifying Unified Domain-Wide Accounts

If a mail account on an external host collects mail directed to all users of your domain, the RPOP module can be instructed to retrieve mail from that account and distribute it to local users.

Unified Domain-wide Accounts
Poll Every Account at Host Password Leave APOP TLS Special Field Last
Not Yet
Poll Every
This option specifies how often the RPOP module should poll the remote account. Set this option to ---- to remove the remote account record. If you set this option to disabled, the account record is not removed, but the remote account is not polled.
Account
This option specifies the name of the mail account on the remote host. For Unified Domain-Wide Accounts, this name is usually your domain name or part of your domain name.
at Host
This option specifies the exact name of the POP server that should be polled. Please note that this could be the name of a specific computer (as specified in DNS A-records), not just a generic domain name of the provider system. For example, if the provider has the domain name provider.com, its POP server is usually named mail.provider.com or pop.provider.com. Consult with your provider.

Standard POP servers accept incoming connections on the TCP port 110. If you need to poll an account on a remote POP server that uses a non-standard port, specify the port number after the host name, using the colon (:) symbol as the separator:
pop.provider.com:111

Password
The password to use to log into the remote account.
Leave
If this option is selected, the RPOP module does not delete messages from the remote account mailbox. Instead, it remembers the UID (Unique IDentifier) of the retrieved messages, and the next time the RPOP module polls this remote account, it does not retrieve messages that have the same UIDs.
If you want to use this option, verify that the remote POP server supports the UIDL command.
APOP
If this option is selected AND the UseAPOP module option is enabled AND the target host advertises APOP capability in its initial prompt, the RPOP module uses the secure APOP method for authentication on that remote host.
TLS
If this option is selected, the RPOP module tries to establish a secure (SSL/TLS) connection with the remote host.

Standard POP servers accept incoming secure connections on the TCP port 995. If you need to poll an account on a remote secure POP server that uses a non-standard port, specify the port number after the host name, using the colon (:) symbol as the separator:
pop.provider.com:9786

Special Field
The name of the messages header (RFC822) field that the provider host inserts into the messages stored in the Unified Domain-Wide Account (see below).

There is always an empty row in the Unified Accounts table. Use it to specify a new Unified Account. To remove an account, set the Poll Every option to -----.

Click the Update button to modify the RPOP module list of the Unified Domain-Wide Accounts.

Special Header Fields and Mail Distribution

When a message is sent via the Internet, the information about the sender and the message recipients is sent in the so-called mail envelope. If mail is sent via SMTP, the envelope is sent as a sequence of the protocol commands, if mail is sent via UUCP, the envelope is sent using additional files.

The information in the envelope is usually the same as the information in the message headers, but it is not always true. The most important exceptions are:

When a message is stored in a mailbox, the envelope information about the sender is added to the message headers as the Return-Path header field. Usually, the envelope information about the recipients is not added to the message headers.

When the RPOP module retrieves a message from a Unified Domain-Wide Account, it has to recompose the message envelope and deliver the message to its final recipient. If the message contains the Return-Path header field, the address in that field is placed in the new envelope as the sender's address, and the header field is removed from the message (it will be recreated when the message is delivered to its final destination).

If a Unified Domain-Wide Account is created with the mail system that can copy the recipient addresses from the envelope into some message header field, then the delivery via RPOP is as reliable as SMTP delivery.
Enter the name of that header field into the Unified Account settings, and the RPOP module will look for that field in all messages retrieved from that account. The addresses from that field will be placed into the new envelope and the messages will be directed to those addresses. The header field itself is removed from the message. All accepted addresses get the 'report on failure' flags, so if message delivery fails, the original message sender (the address in the message Return-Path field) will receive an error report.

A CommuniGate Pro Server can be used to provide Unified Domain-Wide Accounts. For messages stored in those accounts, the envelope recipients are added to the message headers as the X-Real-To fields. To learn how to provide Unified Domain-Wide Accounts with CommuniGate Pro, check the Local Delivery module section.

A legacy sendmail system can be configured to add X-Real-To header fields, too. See the Appendix A below.

Mail Distribution without Special Header Fields

Many ISPs still use various legacy mail systems that cannot store envelope recipients in message headers. If you have to host your Unified Domain-Wide Account on such a system, leave the Special Field empty.

The RPOP module will search for all To:, Cc:, and Bcc: header fields in retrieved messages. It will use the addresses from those header fields only if that address is routed to any existing local CommuniGate Pro Account.

If an address is routed to the SMTP or some other module, or an address cannot be routed at all (unknown user name error, etc.), the RPOP module does not send any error messages to the sender. The module simply ignores that address.

All accepted addresses get the 'do not report failures' flags, so if the message delivery fails for any reason, no error report is sent to the original message sender.

If none of the message To:, Cc:, or Bcc: addresses has been accepted, the RPOP module sends that message to the postmaster Account in the Main Domain.

As explained above, the method based on To:/Cc: header field parsing can cause problems when the actual envelope addresses are not the same as the header field addresses. Besides, some systems do not process the Unified Accounts correctly, so if a message is sent to three users in your domain, those systems may store three copies of the message in the Unified Domain-Wide Account mailbox. Since each message header contains the addresses of all three users, the RPOP module will deliver three copies of the message to each user.

The problems with Bcc, mailing lists, and duplicated message can be very annoying, so we strongly recommend you to ensure that the provider's mail system adds envelope information to the messages stored in your Unified Domain-Wide Account, so you can use the Special Header Field feature.

Specifying Remote Accounts for Individual Users

The CommuniGate Pro RPOP module can poll POP accounts on remote hosts on behalf of the CommuniGate Pro users (Accounts). For each CommuniGate Pro user several external POP accounts can be specified. External accounts can be specified by the Server administrator, via a link on the Account settings page, or by the users themselves, via the WebUser Interface, if the right to specify remote POP accounts has been granted to the user.

Poll Every Account at Host Password Leave APOP TLS Last
12:34:56
 

The settings are the same as for the Unified Accounts, but the Special Header field is not presented. All messages retrieved on the user behalf are directed to that user, regardless of the message header contents.

Last
If the last attempt to retrieve mail from the remote account was successful, this field tells when (in the server local time) this attempt took place.
If the last attempt was not successful, the field contains the error code.

All messages retrieved for inidivdual CommuniGate Pro Accounts are sent to those Accounts via the CommuniGate Pro Queue, so all Server-Wide and Account-Level Rules are applied to those messages.

All messages retrieved for inidivdual CommuniGate Pro accounts get the 'do not report failure' flags, so if delivery was unsuccessful, no error report is sent to the original message sender.

Appendix A. Configuring sendmail for Unified Domain-Wide Accounts

The following file can be used to force the freeware sendmail program to store the envelope information in message headers.

# This file should be placed into the directory cf/feature from
# the sendmail.8.X.XX.cf.tar.Z archive.
# To add special headers, the macros `FEATURE(xrealto)' should be
# added to the main configuration file in the directory cf/cf,
# and the flag T should be added to the mailer description.
#
# This file adds special headers with the `X-Real-To' keyword.
# The special headers will be added to all messages routed to the
# mailer marked with the `T' flag in the sendmail configuration.
divert(0)
VERSIONID(`@(#)xrealto.m4 0.1 1/4/96')
 
divert(9)
# add the X-Real-To: header field to the message
# if the mailer is marked with the `T' flag
H?T?X-Real-To: $u
divert(0)

After these updates are applied, make sure that sendmail delivers all mail for your domain to one account on the sendmail system. The sendmail configuration for that unified account should list the 'mailer' marked with the 'T' flag.

CommuniGate® Pro Guide. Copyright © 1998-2009, Stalker Software, Inc.