Use Threat Intelligence and Python Scripts In Malware Detection
Hi Medium! Here we are again with a new article, today we will talk about advanced threat detection using threat intelligence and well-dedicated scripts using python as follow:
- Threat Intelligence definition
- Blacklisted IP
- Blacklisted Domain
- Blacklisted Certificate
1. Threat Intelligence definition
Intelligence, as defined by Edward Waltz, is “the information and knowledge about an adversary obtained through observation, investigation, analysis, or understanding, is the product that provides battlespace awareness”.
Intelligence is the collecting and processing of information about a competitive entity and its agents, needed by an organization or group for its security and well-being. (from Bimfort’s definition)
Intelligence is defined by Robert Clark as being actionable information.
⇾ There are two key takeaways from these definitions that also apply to CTI. First, intelligence is not just information or data it is information that has been analyzed. Second, intelligence must be actionable. If it is not actionable, there is no benefit to having it.
- Classic Intelligence is as following :
◦ HUMINT: interpersonal
◦ GEOINT: from satellites
◦ MASINT: from radar signatures, nuclear detonation signatures…
◦ SIGINT: cell phone communications or tapping of communication lines
◦ OSINT: form libraries, public records, or the internet
+ OSINT provides too many resources, but they not all valuable as too many feeds repeats, and they are not updated, it’s so important to know how to choose from where you will extract the threat Intel based on your needs.
Cyber threat intelligence (CTI) is a threat of intelligence related to computers, networks, and information technology.
◦ Gartner defines threat intelligence as “evidence-based knowledge, including context, mechanisms, indicators, implications and actionable advice, about an existing or emerging menace or hazard to assets that can be used to inform decisions regarding the subject’s response to that menace or hazard.”
2. Blacklisted IP (C&C, specific malware, spammer, web crawler)
The better feeds of OSINT depends on our needs, in our case we will need to get blacklisted IP for Command and Control server, for most common malware, spammer, and web crawler servers, those feeds need to be in a .csv file, so it will be easier to extract information. After looking on many websites the best one, where we download the file each 5 min so our database will be updated. After we collect all the files we merge them, and we extract only the important data. Each packet we receive on our servers will be validated either the IP used is blacklisted or not, if it’s our client will receive a notification to block the connection.
+ This code will prepare a list of blacklisted IP collected from the different platforms of threat Intel that contain (c2 server, spammer, web crawler, spyware, and other malware).
# Open files
#Split lines and collect all the feeds in one file
for line in c2:
ip = line.split(‘,’)
for line in bl:
ip = line.split()
Before executing the code we need to add a cron that will import the list of blacklisted IP each 5 min with the following command:
Open Cron configuration
# sudo crontab -e
Add the following lines to get file each 5 min
*/5 * * * * cd && wget http://osint.bambenekconsulting.com/feeds/c2-ipmasterlist.txt
*/5 * * * * cd && wget https://myip.ms/files/blacklist/general/latest_blacklist.txt
*/5 * * * * cd && wget http://reputation.alienvault.com/reputation.data
*/5 * * * * cd && python ./merge.py
+ In the following code, we will use the file generated of blacklisted IP and check if the IP captured by tshark by the following fields “ip.dst” and “ip.src” are blacklisted or not.
The Ip is the field received in the frame we will need to scan the external IP
we will use here the file merged with the code “merge.py”
file = open(“reputation.data”)
s = mmap.mmap(file.fileno(), 0, access=mmap.ACCESS_READ)
if s.find(IP) != -1:
print “This “+IP+” is blacklisted”
3. Blacklisted Domain
Same as blacklisted IP, OSINT offer a list of the blacklisted domain used in different attacks, we also download the file each 5 min, and we check if this domain is the packet received from our client, we also check if that domain used in the certificate using the SSL field “x509ce.dNSName”.
Same us IP our client will be notified to block the following domain from reaching his network.
4. Blacklisted Certificate
Each certificate has a unique fingerprint we can calculate it using the hash sha-1, We developed a script that extracts the certificate from the packets received and it automatically calculates the fingerprint.
OSINT provides a list of the fingerprint of certificates used in cyberattacks, and with the following script check if the certificate extracted from the client traffic is blacklisted or not, if they are a specific alert will notify the client to block the malicious connection.
+ This code will take the field “ssl.handshake.certificate” with the tshark command, and check if it is blacklisted or not :
from OpenSSL.crypto import load_certificate, FILETYPE_PEM
#, First of all, we need to add ssl.handshake.certificate to our tshark command
# We need to split the field received as we can receive more than one certificate
# We suppose that we received field B as follow
# Split B in a list
# Function that will transform the certificate from Base64 to Hex and in certificate format and calculate
# the fingerprint
for i in HEX_list:
b = a.decode(“Hex”).encode(“Base64”)
certificate = “ — — -BEGIN CERTIFICATE — — -\n” + b + “ — — -END CERTIFICATE — — -”
cert = load_certificate(FILETYPE_PEM, certificate)
sha1_fingerprint = cert.digest(“sha1”).lower().replace(“:”,””)
#, we need to set a cron that will download the blacklist.csv file each 5 min
# Here is the link https://sslbl.abuse.ch/blacklist/sslblacklist.csv
# Compare with the database
for line in blacklist:
for i in a:
if i == sha1:
print “Evil Match”
# We can also set the file each time we download it with this function to be easier and we got only the needed hash
for line in blacklist:
Before executing the code we need to add a cron that will import the list of blacklisted fingerprint each 5 min with the following command:
Open cron configuration
# sudo crontab -e
Add the following lines, get file each 5 min
*/5 * * * * cd && wget https://sslbl.abuse.ch/blacklist/sslblacklist.csv
In this article, we focused on using threat intelligence to detect attacks using python scripts, those scripts can be implemented in different ways in order to secure our network from external sophisticated attacks.
I hope that you enjoyed the article, don’t hesitate to write a comment if you have something to add !!