[GoLUG] Piping, redirection and shellscipts: 3/5/2025 7pm Eastern Standard time

[Ron]

Steve Litt wrote on 2025-03-02 14:58:

> Yesterday I was tasked with
> getting every domain name I owned into my list at
> 444domains.com/domains , which is created by a shellscript and Python
> program that read a Yaml file and convert it to the web page. So I had
> to do the following:
> 
> * Screen scrape all my domain names from ionos, put in file
> 
> * Screen scrape all my domain names from Spaceship.Com, put in another
>    file
> 
> * Use Vim to remove everything but the domains themselves
> 
> * Sort both files
> 
> * Create a Python program to sort my Yaml file
> 
> * Run diff between each file and the sorted domains of the Yaml file
>      - The output with the left arrows need to be added to Yaml file
> 
> * Re-sort the Yaml file
> 
> * Run the shellscript to create the web page.
> 
> This sounds straightforward

No, it does *not* sound straightforward!


Is it just me, or does it sound like a Rube Goldberg contraption in 
software?



The list of all domain info should be in a DB or at least a text file, 
for starters.


 From there, do the sort(s), build the YAML, etc. Why sort a YAML 
file?!? Running `diff` and manually merging?


All this scraping seems pointless, why is the source of this data inside 
web pages?