Scraping Logos website for "Dynamic Sale Price" and "Dynamic Collection Value" of Logos packages

1Cor10 31

Hi everyone,

Is there a way to pull the "Dynamic Sale Price" and "Dynamic Collection Value" for all the Logos 10 packages into an Excel file automatically (instead of inputting them one by one)?

In the first column, I could write down the name of Logos packages in the first column

In the second column, I could give the weblink for the Logos package.

In the third column, I would like a query of some sort that will use the weblink in Column 2 to grab the Dynamic Sale Price and drop it into Column 3.

In the fourth column, I would like to use a similar query that will use the weblink in Column 2 to grab now the Dynamic Collection Value and drop it into Column 4.

Is the above possible? If so, any tips?

Thanks

Wolfgang Schneider

1Cor10 31 said:

Is the above possible? If so, any tips?

I don't think this is possible ... because the "Dynamic pricing" is dependent on what the person ordering a package already owns in books and features (my current understanding from various previous communications about "you don't pay for what you already own"). So then, 5 customers with different currently owned books/features could have a different "dynamic price" when purchasing the same item/package.

1Cor10 31

https://community.logos.com/discussion/comment/1300485#Comment_1300485

Thank you Wolfgang.

I thought that the whole point of scraping is that the code just picks the number from the website. So if a different person uses the code, the person's own dynamic price should be picked up by the code. 

Bottom line: As a non-tech person, I am not able to see why the fact that the price differs based on the person should prevent us from extract the data. You can simply say - that's the way it works  - and I'm happy to move on from this.

DMB

https://community.logos.com/discussion/comment/1300522#Comment_1300522

Like Wolfgang, not an expert. But your scraping has allow the scripts to fill out the page. Chances are, you're looking at the html delivery? The other issue I run into, is I turn off images, and quite a few page writes sit on images.

Manuel R.

https://community.logos.com/discussion/comment/1300522#Comment_1300522

1Cor10 31 said:

I thought that the whole point of scraping is that the code just picks the number from the website. So if a different person uses the code, the person's own dynamic price should be picked up by the code. 

That's definitely possible with the important remark, that you would need to give the script/crawler the Login information or a valid login token to be able to "see" the corresponding dynamic price and scrape the data.

1Cor10 31

https://community.logos.com/discussion/comment/1300584#Comment_1300584

Hi DMB and Manuel,

Would any of you know the script I have to write? Or point to where I might be able to read and write myself (I'm doubtful I have the skill, but I'll give it a try).

Thanks

Manuel R.

https://community.logos.com/discussion/comment/1300585#Comment_1300585

I know it and could do it but I am sorry I don't have the time to do so. As a developer I am pretty busy. I don't know whether it makes much sense to try it and not understand web-programming / html. You would need python and for example selenium and an html parser like beautiful soup. There are ready to use (often paid) web-crawler / scraper but you would need to configure them for this special use case.

1Cor10 31

https://community.logos.com/discussion/comment/1300785#Comment_1300785

Manuel R. said:

I know it and could do it but I am sorry I don't have the time to do so. As a developer I am pretty busy. I don't know whether it makes much sense to try it and not understand web-programming / html. You would need python and for example selenium and an html parser like beautiful soup. There are ready to use (often paid) web-crawler / scraper but you would need to configure them for this special use case.

Thank you Manuel for responding. I thought it will be a line of code to pull something out like that. You've given me enough pointers to try and experiment.

abondservant

https://community.logos.com/discussion/comment/1300794#Comment_1300794

When we used to fill out a spreadsheet with all the base package info, we did it all by hand. 

1Cor10 31

https://community.logos.com/discussion/comment/1300948#Comment_1300948

I did that too after Logos 10 release, but I was trying to automate this.