Use Threat Intelligence and Python Scripts In Malware Detection

Source

Use Threat Intelligence and Python Scripts In Malware Detection

[4min Reading]

Hi Medium! Here we are again with a new article, today we will talk about advanced threat detection using threat intelligence and well-dedicated scripts using python as follow:

1. Threat Intelligence definition
2. Blacklisted IP
3. Blacklisted Domain
4. Blacklisted Certificate

1. Threat Intelligence definition

Intelligence, as defined by Edward Waltz, is “the information and knowledge about an adversary obtained through observation, investigation, analysis, or understanding, is the product that provides battlespace awareness”.

Intelligence is the collecting and processing of information about a competitive entity and its agents, needed by an organization or group for its security and well-being. (from Bimfort’s definition)

Intelligence is defined by Robert Clark as being actionable information.

⇾ There are two key takeaways from these definitions that also apply to CTI. First, intelligence is not just information or data it is information that has been analyzed. Second, intelligence must be actionable. If it is not actionable, there is no benefit to having it.

• Classic Intelligence is as following :

◦ HUMINT: interpersonal

◦ GEOINT: from satellites

◦ MASINT: from radar signatures, nuclear detonation signatures…

◦ SIGINT: cell phone communications or tapping of communication lines

◦ OSINT: form libraries, public records, or the internet

+ OSINT provides too many resources, but they not all valuable as too many feeds repeats, and they are not updated, it’s so important to know how to choose from where you will extract the threat Intel based on your needs.

Cyber threat intelligence (CTI) is a threat of intelligence related to computers, networks, and information technology.

◦ Gartner defines threat intelligence as “evidence-based knowledge, including context, mechanisms, indicators, implications and actionable advice, about an existing or emerging menace or hazard to assets that can be used to inform decisions regarding the subject’s response to that menace or hazard.”

2. Blacklisted IP (C&C, specific malware, spammer, web crawler)

The better feeds of OSINT depends on our needs, in our case we will need to get blacklisted IP for Command and Control server, for most common malware, spammer, and web crawler servers, those feeds need to be in a .csv file, so it will be easier to extract information. After looking on many websites the best one, where we download the file each 5 min so our database will be updated. After we collect all the files we merge them, and we extract only the important data. Each packet we receive on our servers will be validated either the IP used is blacklisted or not, if it’s our client will receive a notification to block the connection.

+ This code will prepare a list of blacklisted IP collected from the different platforms of threat Intel that contain (c2 server, spammer, web crawler, spyware, and other malware).

# Open files

c2=open(“c2-ipmasterlist.txt”,”r”)

bl=open(“latest_blacklist.txt”,”r”)

reputation=open(“reputation.data”,”a”)

#Split lines and collect all the feeds in one file

for line in c2:

ip = line.split(‘,’)

reputation.write(ip[0]+ “,c2\n”)

for line in bl:

ip = line.split()

try:

reputation.write(ip[0]+ “\n”)

except:

pass

Before executing the code we need to add a cron that will import the list of blacklisted IP each 5 min with the following command:

Open Cron configuration

# sudo crontab -e

Add the following lines to get file each 5 min

*/5 * * * * cd && wget http://osint.bambenekconsulting.com/feeds/c2-ipmasterlist.txt

*/5 * * * * cd && wget https://myip.ms/files/blacklist/general/latest_blacklist.txt

*/5 * * * * cd && wget http://reputation.alienvault.com/reputation.data

*/5 * * * * cd && python ./merge.py

+ In the following code, we will use the file generated of blacklisted IP and check if the IP captured by tshark by the following fields “ip.dst” and “ip.src” are blacklisted or not.

#!/usr/bin/python

import mmap

“””

The Ip is the field received in the frame we will need to scan the external IP

we will use here the file merged with the code “merge.py”

“””

# Open the Ip reputation file and detect blacklisted IP in the traffic

file = open(“reputation.data”)

IP =’207.241.231.146'

s = mmap.mmap(file.fileno(), 0, access=mmap.ACCESS_READ)

if s.find(IP) != -1:

print “This “+IP+” is blacklisted”

file.close()

3. Blacklisted Domain

Same as blacklisted IP, OSINT offer a list of the blacklisted domain used in different attacks, we also download the file each 5 min, and we check if this domain is the packet received from our client, we also check if that domain used in the certificate using the SSL field “x509ce.dNSName”.

Same us IP our client will be notified to block the following domain from reaching his network.

4. Blacklisted Certificate

Each certificate has a unique fingerprint we can calculate it using the hash sha-1, We developed a script that extracts the certificate from the packets received and it automatically calculates the fingerprint.

OSINT provides a list of the fingerprint of certificates used in cyberattacks, and with the following script check if the certificate extracted from the client traffic is blacklisted or not, if they are a specific alert will notify the client to block the malicious connection.

+ This code will take the field “ssl.handshake.certificate” with the tshark command, and check if it is blacklisted or not :

#!/usr/bin/python

import re

from OpenSSL.crypto import load_certificate, FILETYPE_PEM

#, First of all, we need to add ssl.handshake.certificate to our tshark command

# We need to split the field received as we can receive more than one certificate

# We suppose that we received field B as follow

B=”30:82:06:86:30:82:….”

# Split B in a list

HEX_list=B.split(‘,’)

# Function that will transform the certificate from Base64 to Hex and in certificate format and calculate

# the fingerprint

def fingerprint(HEX_list):

fingerprints=[]

for i in HEX_list:

a=str(i).replace(“:”,””)

b = a.decode(“Hex”).encode(“Base64”)

certificate = “ — — -BEGIN CERTIFICATE — — -\n” + b + “ — — -END CERTIFICATE — — -”

cert = load_certificate(FILETYPE_PEM, certificate)

sha1_fingerprint = cert.digest(“sha1”).lower().replace(“:”,””)

fingerprints.append(sha1_fingerprint)

return fingerprints

#, we need to set a cron that will download the blacklist.csv file each 5 min

# Here is the link https://sslbl.abuse.ch/blacklist/sslblacklist.csv

# Compare with the database

def compare(a):

blacklist=open(“sslblacklist.csv”,”r”)

for line in blacklist:

sha1=line.split(‘,’)

try:

for i in a:

if i == sha1[1]:

print “Evil Match”

else:

print “Safe”

except:

pass

blacklist.close()

# We can also set the file each time we download it with this function to be easier and we got only the needed hash

def triage():

blacklist=open(“sslblacklist.csv”,”r”)

blacktriage=open(“blacktriage.csv”,”w”)

for line in blacklist:

sha1=line.split(‘,’)

try:

blacktriage.write(sha1[1]+ “\n”)

except:

pass

blacklist.close()

blacktriage.close()

Before executing the code we need to add a cron that will import the list of blacklisted fingerprint each 5 min with the following command:

Open cron configuration

# sudo crontab -e

Add the following lines, get file each 5 min

*/5 * * * * cd && wget https://sslbl.abuse.ch/blacklist/sslblacklist.csv

Summary

In this article, we focused on using threat intelligence to detect attacks using python scripts, those scripts can be implemented in different ways in order to secure our network from external sophisticated attacks.

I hope that you enjoyed the article, don’t hesitate to write a comment if you have something to add !!