Overview:
The AI-Powered Job Recommendation System will recommend jobs to users based on their skills, experience, and preferences. It will analyze the user’s resume or input data, compare it with available job postings, and suggest the most relevant jobs.
Step-by-Step Guide:
1. Setting Up the Development Environment
Before we start coding, we need to set up a Python environment and install the necessary libraries.
- Create a virtual environment:
# On Windows python -m venv job-recommendation-env job-recommendation-env\Scripts\activate # On macOS/Linux python3 -m venv job-recommendation-env source job-recommendation-env/bin/activate - Install necessary libraries:
pip install flask pandas scikit-learn numpy nltkFlask: Web framework for building the app.Pandas: Data manipulation.Scikit-learn: For machine learning models and recommendations.NLTK: For text processing (to handle resumes and job descriptions).
2. Data Collection and Preprocessing
You will need two datasets:
- User Profile: This is the user’s resume or input data (skills, experience, etc.).
- Job Postings: A list of available job postings with descriptions, required skills, experience, etc.
For simplicity, you can use a sample dataset. Here’s an example of how a sample dataset can look like for job postings:
job_id,job_title,company_name,job_description,skills_required,experience_required
1,Software Engineer,TechCorp,Develop and maintain web applications,Python,JavaScript,3 years
2,Data Scientist,DataInc,Analyze and interpret complex data,Python,SQL,2 years
3,Product Manager,ProdCo,Manage product life cycle,Agile,Leadership,5 years
We can store this data in a CSV file (e.g., job_postings.csv) or a database.
- User Profile Example:
- Skills: Python, JavaScript, SQL
- Experience: 3 years
3. Processing the Data
We need to process the text data (job descriptions and user resumes) to extract features such as skills and experience. You can use TF-IDF (Term Frequency-Inverse Document Frequency) to convert text data into numerical form.
- Job Postings Preprocessing:
- Read the CSV file.
- Preprocess the job descriptions and skills using TF-IDF or a simple bag-of-words approach.
import pandas as pd
from sklearn.feature_extraction.text import TfidfVectorizer
# Load job postings
job_postings = pd.read_csv('job_postings.csv')
# Preprocess job descriptions using TF-IDF
vectorizer = TfidfVectorizer(stop_words='english')
X = vectorizer.fit_transform(job_postings['job_description'])
# Example: Output feature matrix (you can print this to see the features)
print(X.shape)
- User Profile Preprocessing:
- Extract skills and experience from user input (resume).
- For simplicity, assume users provide their skills as a comma-separated list.
def process_user_profile(user_profile):
# For now, assume user profile contains a comma-separated list of skills
skills = user_profile['skills'].split(',')
return skills
# Example user profile
user_profile = {'skills': 'Python,JavaScript,SQL'}
user_skills = process_user_profile(user_profile)
4. Building the Recommendation Model
Now, we will build a simple recommendation model. We’ll compare the user’s skills to job descriptions using cosine similarity, which measures how similar two documents are.
- Calculating Cosine Similarity:
- Cosine similarity will help us identify which job postings are most relevant to the user’s profile based on skills and experience.
from sklearn.metrics.pairwise import cosine_similarity
# Preprocessing user skills (convert into a format compatible with the TF-IDF vectorizer)
user_skills = ' '.join(user_skills) # Joining skills as a string
user_profile_tfidf = vectorizer.transform([user_skills]) # Transform to TF-IDF
# Calculate cosine similarity between user profile and job postings
similarities = cosine_similarity(user_profile_tfidf, X)
# Get job recommendations
recommended_jobs = job_postings.iloc[similarities.argsort()[0][-5:]] # Top 5 recommended jobs
print(recommended_jobs)
5. Building the Web Application (Frontend + Backend)
Next, we’ll integrate the recommendation system into a Flask web app. The user will enter their skills, and the app will recommend jobs based on their input.
Backend: Flask API
- Create a file
app.pyfor your Flask backend:
from flask import Flask, request, jsonify
import pandas as pd
from sklearn.feature_extraction.text import TfidfVectorizer
from sklearn.metrics.pairwise import cosine_similarity
app = Flask(__name__)
# Load job postings
job_postings = pd.read_csv('job_postings.csv')
# Preprocess job descriptions using TF-IDF
vectorizer = TfidfVectorizer(stop_words='english')
X = vectorizer.fit_transform(job_postings['job_description'])
@app.route('/get_job_recommendations', methods=['POST'])
def get_job_recommendations():
user_profile = request.json # Get user profile (skills in this case)
user_skills = user_profile['skills']
# Convert user skills into TF-IDF format
user_skills_str = ' '.join(user_skills) # Joining skills as a string
user_profile_tfidf = vectorizer.transform([user_skills_str])
# Calculate cosine similarity between user profile and job postings
similarities = cosine_similarity(user_profile_tfidf, X)
# Get top 5 recommended jobs
recommended_jobs = job_postings.iloc[similarities.argsort()[0][-5:]]
job_list = recommended_jobs.to_dict(orient='records')
return jsonify({'recommended_jobs': job_list})
if __name__ == "__main__":
app.run(debug=True)
Frontend: HTML + JavaScript
Create a file index.html for the frontend.
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>Job Recommendation System</title>
</head>
<body>
<h1>Job Recommendation System</h1>
<label for="skills">Enter your skills (comma separated):</label>
<input type="text" id="skills" placeholder="e.g., Python, JavaScript, SQL">
<button onclick="getRecommendations()">Get Recommendations</button>
<ul id="job-list"></ul>
<script>
function getRecommendations() {
var skills = document.getElementById("skills").value.split(",");
fetch('/get_job_recommendations', {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({ skills: skills })
})
.then(response => response.json())
.then(data => {
var jobList = data.recommended_jobs;
var listElement = document.getElementById("job-list");
listElement.innerHTML = ''; // Clear previous recommendations
jobList.forEach(job => {
var li = document.createElement("li");
li.textContent = `${job.job_title} at ${job.company_name}`;
listElement.appendChild(li);
});
});
}
</script>
</body>
</html>
6. Running the App
To run the Flask app:
- Ensure you have the
app.pyandindex.htmlfiles in the correct folder. - In the terminal, run:
python app.py
- Open a browser and go to
http://127.0.0.1:5000. You’ll be able to enter skills and get job recommendations.
7. Deploying the Application
Once everything works locally, you can deploy your application to a cloud platform like Heroku or AWS.
- Deploy on Heroku:
- Install the Heroku CLI.
- Create a
Procfileto specify how to run your app:web: python app.py - Push the app to Heroku by following the Heroku deployment guide.
Conclusion
Now, you have a Job Recommendation System built using Flask, TF-IDF, and cosine similarity. You can enhance it further by integrating more advanced machine learning models, adding authentication, or improving the user interface.
