Buckets
Intelligence X categorizes data sources into bucket. Buckets can be used to filter search results and broad identify the source of individual results. For example, the bucket “darknet.tor” indicates the result origins from some a Tor hidden service (onion domains) and was collected by our Tor crawler. Buckets have machine readable names (like “darknet.tor”) and corresponding human readable names (like “Darknet Tor”).
Access to some buckets is restricted to paid licenses, while some may be available as partial preview, and others may be available fully redacted, depending on your license.
You can use buckets as filters to only search specific buckets. Specifying “web.public.com” in the buckets filter (for the Search API endpoint /intelligent/search) will only search in that bucket.
Below is a non-exhaustive list of the most important buckets available. Buckets may be added and remove at any time.
| Bucket | Info | TLDs |
|---|---|---|
| darknet.i2p | I2P Darknet, .i2p sites | i2p |
| darknet.tor | Tor Darknet, .onion sites | onion |
| dns | DNS data from domains | |
| documents.public.scihub | Documents Public: Sci-Hub | |
| dumpster | Dumpster: Everything that does not fit anywhere else | |
| dumpster.web.1 | Dumpster: Websites | |
| dumpster.web.ssn | Dumpster: SSN related websites | |
| leaks.logs | Stealer Logs: Data collected and leaked by viruses | |
| leaks.private.general | Restricted Leaks | |
| leaks.public.general | Public Leaks | |
| leaks.public.wikileaks | WikiLeaks: Published leaked emails and documents, Snowden Documents, Cryptome | |
| pastes | Pastes | |
| patent.us | Patents: US | |
| usenet | Usenet | |
| web.gov.ru | Public Web: Russia Government | gov.ru |
| web.public.af | Public Web | ac, dz, ao, bj, bw, bf, bi, cm, cv, cf, td, km, cd, dj, eg, gq, er, sz, et, ga, gm, gh, gn, gw, ci, ke, ls, lr, ly, mg, mw, ml, mr, mu, yt, ma, mz, na, ne, ng, cg, rw, re, sh, sn, sc, sl, so, za, ss, sd, st, tz, tg, tn, ug, eh, zm, zw |
| web.public.ams | Public Web | ag, ai, ar, aw, bb, bm, bo, bq, br, bs, bz, ca, cl, co, cr, cu, cw, dm, do, ec, fk, gd, gf, gp, gt, gy, hn, ht, jm, kn, ky, lc, mq, ms, mx, ni, pa, pe, pr, py, sr, sv, sx, tc, tt, us, uy, vc, ve, vg, vi |
| web.public.aq | Public Web | aq |
| web.public.business | Public Web | biz, icu, shop, work, site |
| web.public.cee | Public Web | al, ba, bg, by, cz, ee, gr, hr, hu, lt, lv, md, me, mk, pl, ro, rs, si, sk |
| web.public.cn | Public Web | cn |
| web.public.com | Public Web | com |
| web.public.de | Public Web | at, de, lu, li, ch |
| web.public.eu | Public Web | eu |
| web.public.gov | Public Web: U.S. Government | gov |
| web.public.info | Public Web | info |
| web.public.kp | North Korea: Public Government Sites | kp |
| web.public.mea | Public Web | af, am, az, bh, bd, bt, bn, kh, cy, ge, hk, in, id, ir, iq, il, jp, jo, kw, kg, la, lb, mo, my, mv, mn, mm, np, om, pk, ph, qa, sa, sg, kr, lk, ps, sy, tw, tj, th, tl, tr, tm, ae, uz, vn, ye |
| web.public.misc | Public Web | top, wang, xyz, рф (xn–p1ai), ä¸å›½ (xn–fiqs8s) |
| web.public.net | Public Web | net |
| web.public.nord | Public Web | uk, ie, is, dk, no, fi, se, gl |
| web.public.oc | Public Web | as, au, cc, ck, fm, fj, pf, gu, ki, mh, nr, nc, nz, nf, mp, pw, pg, pn, ws, sb, to, tv, vu, wf |
| web.public.org | Public Web | org |
| web.public.peer | Public Web: Decentralized blockchain based TLDs | bit, bazar |
| web.public.ru | Public Web: Ukraine | ua, kz, ru |
| web.public.social | Public Web | club, vip |
| web.public.tech | Public Web | app, io, online |
| web.public.we | Public Web | ad, es, fr, it, mc, mt, nl, be, pt, sm, va |
| whois | Whois data |