Google Play Apps Rating Predictions¶
- What kind of Andriod Apps are most likely to get high ratings?
Load Data¶
In [ ]:
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import seaborn as sns
%matplotlib inline
In [ ]:
raw = pd.read_csv("/content/drive/My Drive/Home/bibina Google play/googleplaystore.csv")
In [ ]:
raw.head()
Out[ ]:
In [ ]:
raw.info()
In [ ]:
data = raw.dropna()
EDA¶
Category¶
- First, let's take a look at the top 5 categories with the highest ratings
In [ ]:
data.groupby('Category').mean().sort_values(by='Rating',ascending=False).head(5)
Out[ ]:
In [ ]:
plt.figure(figsize=(16,8))
fig = sns.boxplot(x="Category",y="Rating",data=data,palette = "Set1")
fig.set_xticklabels(fig.get_xticklabels(),rotation=90)
plt.title('Distribution of Ratings in Each Category',fontsize = 20)
Out[ ]:
- From this box plot, we can see that the ratings of "Events" and "Education" categories are all pretty steady. Even the lowest rating is above 3.5.
- Ratings of "Business", "Dating", "Finance", "Lifestyle" and "Tools" categories vary a lot.
Reviews¶
In [ ]:
pd.to_numeric(data.Reviews)
Out[ ]:
In [ ]:
plt.figure(figsize=(10,5))
sns.regplot(x=pd.to_numeric(data.Reviews),y=data.Rating,data=data)
plt.title('Rating vs Reviews',fontsize = 20)
Out[ ]:
- Reviews seem to be slightly correlated to ratings. More reviews, higher ratings.
Size¶
- Take a rough look at the range of size.
In [ ]:
data.Size.unique()
Out[ ]:
- Change 'k','M' and 'Varies with device' with numbers
In [ ]:
data.Size.replace({'Varies with device':np.nan},regex=True,inplace=True)
data.Size = (data.Size.replace(r'[kM]+$', '', regex=True).astype(float) * \
data.Size.str.extract(r'[\d\.]+([KM]+)', expand=False)
.fillna(1)
.replace(['k','M'], [1, 1000]).astype(int))
In [ ]:
plt.figure(figsize=(10,5))
sns.regplot(x=pd.to_numeric(data.Size),y=data.Rating,data=data)
plt.title('Rating vs Size',fontsize = 20)
Out[ ]:
- Size seems seldomly affect ratings, but apps with larger size have higher ratings on average.
Installs¶
In [ ]:
data.Installs.unique()
Out[ ]:
In [ ]:
ins = data.groupby('Installs').mean()
In [ ]:
plt.figure(figsize=(16,8))
fig = sns.barplot(x=ins.index,y=ins['Rating'],data=data,palette = "Set1")
fig.set_xticklabels(fig.get_xticklabels(),rotation=60)
plt.title('Ratings vs Install Times',fontsize = 20)
Out[ ]:
- Apps with very few installs as well as very large number of installs have relatively higher ratings.
Price¶
In [ ]:
data.Price = data.Price.replace(r'[$]+', '', regex=True).astype(float)
In [ ]:
plt.figure(figsize=(10,5))
sns.regplot(x=data.Price,y=data.Rating)
plt.title('Rating vs Price',fontsize = 20)
Out[ ]:
In [ ]:
plt.figure(figsize=(10,5))
sns.regplot(x='Price',y='Rating',data=data[data.Price<=50])
plt.title('Rating vs Price(less than $50)',fontsize = 20)
Out[ ]:
- Apps with higher price tend to have higher ratings. But there are also a lot of free apps getting full star ratings. Let's take a look at them.
In [ ]:
fivestar = data[(data.Price==0)&(data.Rating==5.0)]
In [ ]:
fivestar.shape
Out[ ]:
In [ ]:
plt.figure(figsize=(16,8))
fig = sns.countplot(x='Category', data=fivestar,palette='Set1')
fig.set_xticklabels(fig.get_xticklabels(),rotation=60)
plt.title('5 Star Free Apps in Each Category ',fontsize = 20)
Out[ ]:
- There are nearly 60 five star free apps in "Family" category. "Medical" and "Lifestyle" category also did pretty well.
Predictive Analysis¶
From previous analysis, I found that around 1500 ratings were missing. So, next, I'm going to use other complete rows of information to predict those missing ratings.
Feature Engineering¶
Now, it's time to choose useful features for predicting models. I will drop some columns including:
- App: ID like, should be excluded from the data
- Last Updated: Intuition. Seem have nothing to do with Ratings
- Current Ver: not consistent
- Type: provide replicated information with Price
- Genres: provide replicated information with Category
In [ ]:
raw.drop(['App','Last Updated','Current Ver','Type','Genres'],axis=1,inplace=True)
In [ ]:
raw = raw[raw.Size!='1,000+']
In [ ]:
raw=raw[raw.Size!='Varies with device']
raw.Size = (raw.Size.replace(r'[kM]+$', '', regex=True).astype(float) * \
raw.Size.str.extract(r'[\d\.]+([KM]+)', expand=False)
.fillna(1)
.replace(['k','M'], [1, 1000]).astype(int))
In [ ]:
raw.Reviews = pd.to_numeric(raw.Reviews)
- Divide features into numerical and categorical ones.
In [ ]:
num_col = [cname for cname in raw.columns if raw[cname].dtype in ['int64','float64']]
In [ ]:
cat_col = [cname for cname in raw.columns if raw[cname].dtype=='object']
In [ ]:
cat_col
Out[ ]:
In [ ]:
num_col
Out[ ]:
- Perform One Hot Encoding on categorical variables.
In [ ]:
raw.head()
Out[ ]:
In [ ]:
new = pd.get_dummies(raw, prefix=cat_col, drop_first=True)
In [ ]:
new
Out[ ]:
- Set target variable: Rating
In [ ]:
#filter out the test set
test = new[new.Rating.isna()]
In [ ]:
label = new.Rating.dropna()
In [ ]:
label
Out[ ]:
In [ ]:
data = new.dropna().drop('Rating',axis=1)
Modeling¶
In [ ]:
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import StandardScaler
- Standard Scaler
In [ ]:
scaler = StandardScaler()
X = scaler.fit_transform(data)
- Train Test Split
In [ ]:
X_train, X_test, y_train, y_test = train_test_split( X, label, test_size=0.2, random_state=101)
- XGBoost Regressor
In [ ]:
from xgboost import XGBRegressor
from sklearn.metrics import mean_absolute_error
In [ ]:
def getAccuracy_cv(model):
model.fit(X_train,y_train)
# get predictions
preds = model.predict(X_test)
# cross validation
from sklearn.model_selection import cross_val_score
scores = -1 * cross_val_score(model,X,label,cv=10,scoring = 'neg_mean_absolute_error')
print(scores.mean())
In [ ]:
# Hyperparameter Tuning
paraList = [500, 1000, 1500]
for i in paraList:
model =XGBRegressor(
learning_rate =0.1,
n_estimators=i,
max_depth=5,
min_child_weight=1,
gamma=0,
subsample=0.8,
colsample_bytree=0.8,
nthread=4,
scale_pos_weight=1,
seed=27)
getAccuracy_cv(model)
In [ ]:
final_model =XGBRegressor(
learning_rate =0.1,
n_estimators=500,
max_depth=5,
min_child_weight=1,
gamma=0,
subsample=0.8,
colsample_bytree=0.8,
nthread=4,
scale_pos_weight=1,
seed=27)
getAccuracy_cv(final_model)
Predict¶
- Repeat the preprocessing steps on test data
In [ ]:
test = test.drop('Rating',axis=1)
test.head()
Out[ ]: