Botify API allows you to list and filter analyzed URLs and return their fields value (metadata). Full list of requestable fields can be found in Analysis Datamodel.
analyses/{username}/{project_slug}/{analysis_slug}/urls
BQLQuery
Pagination<BQLResult>
Please refer to BQLQuery documentation for information about how to define fields to select, filters and sort.
curl "https://api.botify.com/v1/analyses/${username}/${project_slug}/${analysis_slug}/urls" \
-H "Authorization: Token ${API_KEY}" \
-H "Content-type: application/json" \
--data-binary "${BQLQuery}"
The following example of BQLQuery fetches url
and metadata.title.nb
fields and filters the dataset on new URLs that respond with a 2xx HTTP code. The result is sorted on descending number of titles.
{
"fields": [
"url",
"metadata.title.nb"
],
"filters": {
"and": [
{
"field": "http_code",
"predicate": "between",
"value": [200, 300]
},
{
"not": {
"field": "previous",
"predicate": "exists"
}
}
]
},
"sort": [
{
"metadata.title.nb": {
"order": "desc"
}
}
]
}
A sample result would be:
{
"count": 11,
"page": 1,
"size": 2,
"results": [
{
"url": "www.abc.com/1",
"metadata": {
"title": {
"nb": 15
}
}
},
{
"url": "www.abc.com/2",
"metadata": {
"title": {
"nb": 9
}
}
}
]
}