desktop application / script to interact with javascript on web application

Question

My work has tasked me with determining the feasibility of migrating our existing in-house built change management services(web based) to a Sharepoint solution. I've found everything to be easy except I've run into the issue that for each change management issue (several thousand) there may be any number of attachment files associated with them, called through javascript, that need to be downloaded and put into a document library.

(ex. ... onClick="DownloadAttachment(XXXXX,'ProjectID=YYYY');return false">Attachment... ).

To keep me from manually selecting them all I've been looking over posts of people wanting to do similar, and there seem to be many possible solutions, but they often seem more complicated than they need to be.

So I suppose in a nutshell I'm asking what would be the best way to approach this issue that yields some sort of desktop application or script that can interact with web pages and will let me select and organize all the attachments. (Making a purely web based app (php, javascript, rails, etc.) is not an option for me, so throwing that out there now).

Thanks in advance.

the onclick="DownloadAttachment(xxxxx,'ProjectID=yyyy')... was just to show I was dealing with a javascript method and not a direct link to the files needed. To expand on the first part of my issue: The project ID I know (there are 3 Im dealing with) and the individual issue IDs I can generate a report with all the numbers of, which can then be used in url to go to each issue's page. The problem is that on each of these pages there may or may not be any number of links to attachments (the javascript download method) with arbitrary number IDs. — TheBeeKeeper
– TheBeeKeeper, Commented Jun 2, 2011 at 12:01
And after inspecting the javascript "DownloadAttachment()" function (By the way, I just discovered the built in chrome dev tools = amazing) I can assemble a url for each attachment link. So I suppose I need a way to go to pages, inspect their source, and if a download link exists, construct a URL for it, save the attachment, and move on to the next. — TheBeeKeeper
– TheBeeKeeper, Commented Jun 2, 2011 at 12:31

Binil Thomas · Accepted Answer · 2011-06-01 20:01:26Z

1

Given a document id and project id, XXXXX and YYYY respectively in your example, figure out the URL from which the file contents can be downloaded. You can observe a few URL links in the browser and detect the pattern which your web application uses.
Use a tool like Selenium to get a list of XXXXXs and YYYYs of documents you need to download.
Write a bash script with wget to download the files locally and put in the correct folders.

answered Jun 1, 2011 at 20:01

Binil Thomas

13.8k10 gold badges59 silver badges70 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

Darien · Accepted Answer · 2011-06-01 20:07:46Z

This is a "one off" migration, right?

Get access to your in-house application's database, and create an SQL query which pulls out rows showing the attachment names (XXXXX?) and the issue/project (YYYY?), ex:
```
|file_id|issue_id|file_name           |  
|      5|     123|Feasibility Test.xls|
```
Analyze the DownloadAttachment method and figure out how it generates the URL that it calls for each download.
Start a script (personally I'd go for Python) that will do the migration work.
Program the script to connect and run the SQL query, or can read a CSV file you create manually from step #1.
Program the script to use the details to determine the target-filename and the URL to download from.
Program the script to download the file from the given URL, and place it on the hard drive with the proper name. (In Python, you might use urllib.)

Hopefully that will get you as far as a bunch of files categorized by "issue" like:

issue123/Feasibility Test.xls
issue123/Billing Invoice.doc
issue456/Feasibility Test.xls

TheBeeKeeper · Accepted Answer · 2011-06-02 19:53:27Z

0

Thank you everyone. I was able to get what I needed using htmlunit and java to traverse a report I made of all change items with attachments, go to each one, copy the source code, traverse that to find instances of the download method, and copy the unique IDs of each attachment and build an .xls of all items and their attachments.

answered Jun 2, 2011 at 19:53

TheBeeKeeper

2252 silver badges13 bronze badges

Collectives™ on Stack Overflow

desktop application / script to interact with javascript on web application

3 Answers 3

Comments

Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

Comments

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related