1. Before Start

Define your license on begin of main.js script like this:

#LICENSE_KEY "99999-99999-99999-99999-99999"

Below, are defined all scripts included.

//relative to main.js 
#require "./scripts/my_script.js" 
//or 
#require "C:/scripts/my_script.js"

2. Defining Request Headers

The headers can be configured any time during script execution because are saved in a vector on browser and are utilized before each request. Define a header will remove the old header and only change when other header is defined with same name, this allow change user agent for each url whithout break the script.

//userAgent only for request 
scrapbot.setRequestHeader('User-Agent', 'scrapbot');

3. Defining User Agent

Defines user-agent for request and navigator.

//userAgent for request and navigator 
await scrapbot.setUserAgent('scrapbot');

4. Creating a Simple Authentication Script

await scrapbot.geturl("http://www.website.com"); 
 
await scrapbot.selector('input#user', 5000).sendKeys('user'); 
 
await scrapbot.selector('input#password', 5000).sendKeys('password'); 
 
await scrapbot.selector('input#submit', 5000).click();

This script perform an authentication sending a form. The sendKeys method simulates keyboard event on the region where is the input. The click method do same thing with mouse event.

After submitted, the API will wait page response to execute next lines. The javascript API uses promise, so is possible control loading errors. The second parameter "5000" is a wait time to find selector, this prevent time problems unespected.

5. Navigation and Selector

Requests an url from internet. All page content can be accessed directly through scrapbot api. Returns true if successful or false in error case.

await scrapbot.geturl("http://www.website.com"); 
//or 
await scrapbot.geturl("www.website.com");

 

Reload page.

//ignoreCache: true or false. Default: false 
await scrapbot.reload(ignoreCache);

 

Wait for 5 seconds.

await scrapbot.sleep(5000);

 

The selector can be called passing this arguments:

//css selector 
await scrapbot.selector("body > div > div", timeout); 
 
await scrapbot.selector("div > div", object, timeout); 
 
await scrapbot.selector(object); 
 
//or xpath selector 
await scrapbot.selector("//body/div[1]/div[1]", timeout); 
 
await scrapbot.selector("//div[1]/div[1]", object, timeout);

On second method, the parameter "object" is a context for selector. For example, for a specific search inside a element, just pass the element on second argument. This call will be utilized for access elements inside an iframe passing the iframe by parameter: await scrapbot.selector(path, iframeObject, timeout);

On third method, the parameter "object" will be target of a event like click, sendkeys, or other. On this case, will not search by selector, just fire events on parameter passed.

 

Each selector call will return a object contains methods for actions over a element list found.


    focus: function(), 
     
    sendKeys: function(keys), 
     
    click: function(), 
     
    rightClick: function(), 
     
    //returns null if element not exists 
    get: function(index), 
     
    toArray: function(), 
     
    length: function() 
}

 

This methods also can be called directly from api without selector:

scrapbot.focus(x, y); 
 
scrapbot.sendKeys(x, y, keys); 
 
scrapbot.click(x, y); 
 
scrapbot.rightClick(x, y);

On this case, not have promise and is not necessary "await".
The directly access from this methods allows fires events on page regions that do not have DOM elements, like as flash objects.

 

Returns scroll height of page.

scrapbot.scrollHeight();

 

Returns current scroll position. This method and scrollHeight() can be useful for move page scroll and load dynamic elements.

scrapbot.scrollTop(y);

 

Returns software version.

scrapbot.version();

 

Returns an integer greater than 0 that represents the id of current thread, this id is created by each program instance. Useful to control write and read of files to specific threads.

scrapbot.getThreadId();

6. Files and Directories

Returns true if file exists and false if not.

scrapbot.isfile(path);

 

Returns the file size in bytes. If not exists, returns 0.

scrapbot.filesize(path);

 

Returns true if directory exists and false if not.

scrapbot.isdir(path);

 

Saves a file on disk.

scrapbot.savefile(file, data);

 

Read a file from disk.

scrapbot.readfile(file);

 

Copy a file. Returns true if successful or false in error case.

scrapbot.copy(src, dst);

 

Renames files and folders. Returns true if successful or false in error case.

scrapbot.rename(old_path, new_path);

 

Removes a file from disk. Returns true if successful or false in error case.

scrapbot.removefile(path);

 

Removes a directory from disk. This methods removes recursively all files and subdirectories and, for last, removes root directory. Returns true if successful or false in error case.

scrapbot.removedir(path);

 

Creates a empty directory. Returns true if successful or false in error case.

scrapbot.mkdir(path);

 

Returns the current directory.

scrapbot.getCurrentDir()

7. Downloads and Uploads

Performs a file download from internet. Can be useful for PDFs, images, and others. Returns true if successful or false in error case.

await scrapbot.downloadfile(url, path);

 

Sends one or more files to server. The first parameter is the position of file filter that is show on menu dropdown of upload dialog. The second parameter is the DOM element of input file, this is necessary for api define all upload files. The last parameter are all upload files that will be send. ScrapBot will check if each file exists and if any error occur, an exception will be trigger.

//upload multiple files 
await scrapbot.setFileUpload(0, inputFileDom, [".\\relative_path\\test.text", "C:\\test.txt", "C:\\test_2.txt", "C:\\test_3.txt"]); 
 
//upload single file 
await scrapbot.setFileUpload(0, inputFileDom, ["C:\\test.txt"]); 
//or 
await scrapbot.setFileUpload(0, inputFileDom, "C:\\test.txt");

8. Proxy

This method defines proxy configuration. Can be called any time and will persist until next change or until disable proxy.

//enable proxy 
scrapbot.setProxy('http://host:port'); 
 
//disable proxy 
scrapbot.setProxy(false);

9. Run External Programs

This method can run any program or any command of system. Will not return any value and not wait for response.

//program, directory, arguments, hidden 
scrapbot.shellexecute('notepad.exe', null, null, false);

10. Get Window Object

Returns the window object.

scrapbot.getWindow();

11. Screen Capture

Captures a region of page and save to a file.

//x, y, width, height, scale, path 
await scrapbot.screenshot(0, 0, 500, 500, 1, ".\\image.jpg");

12. Get Source Code as String

Returns the page source code or text.

//with html 
await scrapbot.getsource(); 
 
//only text 
await scrapbot.gettext();

13. Session

Clear session data.

//clear cache 
await scrapbot.clearBrowserCookies(); 
 
//clear cookies 
await scrapbot.clearBrowserCache();

 

Cookies handling.

//return array of cookie objects 
await scrapbot.getAllCookies(); 
 
//cookies: array of cookie objects 
await scrapbot.setCookies(cookies);

Cookie object:


    name, //string - Cookie name. 
     
    value, //string - Cookie value. 
     
    domain, //string - Cookie domain. 
     
    path, //string - Cookie path. 
     
    expires, //number - Cookie expiration date as the number of seconds since the UNIX epoch. 
     
    httpOnly, //boolean - True if cookie is http-only. 
     
    secure, //boolean - True if cookie is secure. 
     
    session //boolean - True in case of session cookie. 
}

14. Absolute and Relative Paths

The methods for file system manipulation can works with absolute path or relative path to main.js. A call to save a file can be done like this:

//relative to main.js 
scrapbot.savefile('./test.txt', 'Hello World!'); 
 
//absolute path 
scrapbot.savefile('C:/documents/test.txt', 'Hello World!');