Hello, there is the following task:
There is a txt file with a list of url of the form(100k lines) without http://and https://
site.com/123.php
site2.info/456.php

I need to save in a separate file only those urls in the source code of which pages contain certain text
For example,"Statistic and a Parameter"

I tried to implement this task using a .sh script, because I need to run the script on Ubuntu, but I could not figure it out with curl.
Thank you in advance for your help!
  • What did not work out exactly? In general, something like this should be:

    grep"text""$(curl site.com/123.php)"
    – Modern1 Aug 11 '19 at 00:53
  • It was not possible to open the list of sites in the sh script and pass it as a variable in curl – M.C. Escher's Aug 11 '19 at 01:34
  • M.C. Escher's, So bring your script, see, fix it. – Gifted Grebe Aug 11 '19 at 01:49

1 Answers 1

something like that?
#!/bin/bash
foreach read;do
  echo"processing the following string: ${REPLY}"
  if grep"Statistic and a Parameter""$(curl ${REPLY})";do
    echo"${REPLY}">>urls_with_text.txt
  done
done<file_with_urls.txt