Skip to content

Reddit Urls Scraper

   

Showcasing a command line tool to fetch urls from posts from a given subreddit.

Can set different parameters via command line like saving to a file or STDOUT or limit the number of fetched urls.

The code can be found in https://github.com/miguelabate/reddit-scraper

Also, it’s a good example of how to use the library github.com/devfacet/gocmd to build command line applications

Example call

./reddit-scraper urls -s Pics -l 10 -v

Result:

Request to: https://www.reddit.com/r/Pics/.json?limit=100&after=
Urls fetched: 102
https://www.reddit.com/r/pics/comments/fjn0j9/important_psa_no_you_did_not_win_a_gift_card/
https://www.reddit.com/r/pics/comments/fk5iz1/rpics_is_looking_for_new_moderators_apply_within/
https://i.redd.it/rocsc5h0n0o41.jpg
https://i.imgur.com/yvIuiU8.jpg
https://i.redd.it/i26pv2exb0o41.jpg
https://i.imgur.com/U4B9dYV.jpg
https://i.imgur.com/vis0ggq.jpg
https://i.redd.it/hwjcoj3db1o41.jpg
https://i.redd.it/q3dyobfzcxn41.jpg
https://i.redd.it/3aobmod56xn41.jpg
Total Urls fetched: 10

Related Posts

  1. Protobuf communication between Js-client and Go-server