we write about the things we build and the things we consume
Dragos Pitica

downloading…

Recently, I’ve worked on a component for one of our ingesters. Its scope is to update the data files that we receive from the provider and need to be processed. Therefore, today I’ll show you a couple of basic ways to download one or multiple files in Java.

http

Our use case would be that someone constantly uploads a set of files on a website and we need to create a process to download some files that end with a specific extension. As far as I’m concerned, the easiest way to do that would be if the files are stored on an HTTP server.

The old and less easy way would include BufferedInputStreams/ ReadableByteChannels to read data and FileOutputStreams to write data. Also, in order to filter the files we need to download, we just need to pass in the name of the file requested.



It is a pretty complex way comparing to the next one.

Thankfully, Apache developed a library called Commons IO that provides great facilities to assist with handling files and directories. It takes care of all the stream housekeeping of opening and closing streams and many others.



ftp

On the other hand, we could also remotely download files from an FTP server.

Most of the process is pretty much the same thing as the HTTP one, but it does not require third party FTP tool or application server. However, again, Apache provides Commons Net which is a library that, among other protocols, also implements the client side of FTP. Moreover, for the file name filtering, there’s an FTPFileFilter we can use.



Both approaches are simple and easy to implement into your own application, but what do you think? Have you worked with any of these techniques before or do you use a different one? Let us know in the comments!

If you enjoyed the read, drop us a comment below or share the article, follow us on Twitter or subscribe to our #MetaBeers newsletter. Before you go, grab a PDF of the article, and let us know if it’s time we worked together.

blog comments powered by Disqus
sign up to #metabeers
slideshow