Ask Your Question

How can i download files and folder recursively from HTTP using puppet

asked 2016-11-02 11:27:49 -0600

csepari gravatar image

I wan to download all files and folder located at one particular HTTP path example I need all files and folders from

edit retag flag offensive close merge delete


Puppet is not an HTTP client [well, technically …]. Try to find a better design. Have you considered a network mount? Do you really need all files/directories from that origin? Why doesn't a file resource solve your situation? Q'n'D solution: Write an exec resource which invokes `wget(1)`.

Kai Burghardt gravatar imageKai Burghardt ( 2016-11-07 08:28:15 -0600 )edit

2 Answers

Sort by » oldest newest most voted

answered 2016-11-08 10:39:42 -0600

Kevin T gravatar image

updated 2016-11-08 10:49:46 -0600

if you cant mount directly as the source is coming from somewhere where this cannot be achieved and you think you need to use puppet to fetch these files to the client then as suggested in the comments you would need to find a way of running an exec based on an event. I would suggest that you find away if puppet can quickly check the contents of the top level directoy of the source via some sha1 checksum (like git does for the tree) and if the checksum (1 source file) that you can check does not match then you run Exec resource with an rsync to mirror the source.

The checksum can be generated at the far side when the source files change, as i dont know what your trying to do this is my suggestion. Sync the source to the client based on a single file that is managed by puppet. = rsync over http is not supported but you can use zsync

edit flag offensive delete link more

answered 2016-11-09 08:21:13 -0600

DarylW gravatar image

updated 2016-11-09 09:07:51 -0600

My best suggestion is to write either a custom fact to fetch all of the http paths that you will be pulling down (or a hiera_backendthat could populate a variable with an array of the files you require). In either case you can hit the HTTP api, get all the file paths that you wish to download, then iterate over those files and download them with something like

If you want me to expand on this further, let me know, and I'll try to get to it!

-- EDIT -- Not sure if this backend suits your needs, or if it would be a good place to start..

edit flag offensive delete link more

Your Answer

Please start posting anonymously - your entry will be published after you log in or create a new account.

Add Answer

Question Tools



Asked: 2016-11-02 11:27:49 -0600

Seen: 227 times

Last updated: Nov 09 '16