pcsanwald
|
Posted: October 22, 2010 17:18 by pcsanwald
|
|
I'm wondering if there's a reason Transport doesn't expose a way to send multiple messages, something like: public abstract void sendMessages(List<Tuple<Message, Address>>) the reason I ask is that in my testing, I'm seeing Transport.sendMessage() calls take around 50ms, and ideally I'd like them to be much faster. in further sendmail testing, it seems that waiting for the response code after sending is the slowest part. For a lot of messages, on my postfix instance (using a bash script talking smtp to 25), it's much faster to do all the writes, followed by all the reads, rather than write then read for each message. I timed the two scripts: one that reads the response after every write, another that does all the writes followed by all the reads. the former takes 4s, the latter takes .2s. thoughts? is this a bad idea? |
Transport: multiple messages?
5 posts
Replies: 4 - Last Post: October 25, 2010 12:56
by: pcsanwald
by: pcsanwald
showing 1 - 5 of 5
shannon
|
Posted: October 22, 2010 23:40 by shannon
|
|
Transport.send is for convenience, not speed. If you want to send a lot of messages, you'll want to manage the Transport object yourself. See this FAQ entry for the general approach. The SMTP protocol allows for some amount of pipelining, if the server supports it, but JavaMail doesn't currently take advantage of that - it's much harder to handle the error cases. |
|
pcsanwald
|
Posted: October 23, 2010 19:28 by pcsanwald
|
|
Bill, thanks for responding! the first thing I had initially tried was using managing the Transport object as the FAQ suggests, however, this didn't buy me what I had hoped and so I started investigating a bit more. I actually think that for my specific use case, reading the response after every send in SMTPTransport.java is what's hurting performance the most, since I am using postfix, and it lets you do all the writes, then all of the response reads, rather write/read for each message. I tested this using two bash scripts talking smtp to my local postfix, the first one has a loop (psuedocode as I'm not @ work): # write each message, and read the response code afterwards for (message : messages) { write(message) responseCode = readResponse() } This was pretty slow (4s for 100 emails). next, a colleague suggested batching writes and then doing reads, so we tried: # do all the writes, followed by all the reads. for (message: messages) { write(message) } for(message: messages) { responseCode = readResponse() } this was lightning fast and took .2s! looking at SMTPTransport.java, since it only exposes methods to send one message at a time, obviously the second implementation isn't a viable option. however, if there were a method sendMessages that took multiple messages, then I could implement the "batched" way of processing, write all the messages and then do all the reads, and my email process should speed up a lot. what do you think? |
|
shannon
|
Posted: October 23, 2010 22:04 by shannon
|
|
Remember that each message isn't one write with one response. Each message involves multiple commands and responses. See the SMTP protocol spec for details. And whether some commands are accepted depends on whether previous commands succeeded or failed. If you send the body of the message and the server isn't accepting it, it will try to interpret it as commands and you'll be in a heap of trouble. Pipelining is possible, and the SMTP spec explains where its safe, but it's nowhere near as easy as you imply. |
|
pcsanwald
|
Posted: October 25, 2010 12:56 by pcsanwald
|
|
Bill, thanks again for taking the time to reply. I didn't mean to trivialize pipelining, but after reading the SMTP spec your comments make perfect sense, failure scenarios indeed sound problematic. it seems like the best route for my application to take is paralellizing calls to Transport.sendMessage() to improve send throughput, as I think I've tried just about everything else with not much luck. thanks, --paul |
Replies: 4 - Last Post: October 25, 2010 12:56
by: pcsanwald
by: pcsanwald
