big_data_analysis/data_visualisation
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
data_visualisation/practice3/main.ipynb

6.6 MiB
Raw Permalink Blame History

Практическая работа №3

Вариант 8

Таблица находится в конце файла. Так же все файлы проекта (в том числе отчёт в формате odt или pdf) можно скачать с git: https://git.dm1sh.ru/data-visualization/practice3

Изначально проект писался в vscode под линуксом, если не работает установка пакетов в первом блоке, попробуйте заменить % на ! или установить пакеты вручную и удалить его

  1. Подгружаем нужные для работы пакеты
In [225]:
# установка matplotlib, numpy и pandas
%pip install matplotlib numpy pandas seaborn scikit-learn ChernoffFace --upgrade

Requirement already satisfied: matplotlib in ./.venv/lib64/python3.11/site-packages (3.7.1)
Requirement already satisfied: numpy in ./.venv/lib64/python3.11/site-packages (1.24.2)
Requirement already satisfied: pandas in ./.venv/lib64/python3.11/site-packages (1.5.3)
Requirement already satisfied: seaborn in ./.venv/lib64/python3.11/site-packages (0.12.2)
Requirement already satisfied: scikit-learn in ./.venv/lib64/python3.11/site-packages (1.2.2)
Requirement already satisfied: ChernoffFace in ./.venv/lib64/python3.11/site-packages (0.1.4)
Requirement already satisfied: contourpy>=1.0.1 in ./.venv/lib64/python3.11/site-packages (from matplotlib) (1.0.7)
Requirement already satisfied: cycler>=0.10 in ./.venv/lib64/python3.11/site-packages (from matplotlib) (0.11.0)
Requirement already satisfied: fonttools>=4.22.0 in ./.venv/lib64/python3.11/site-packages (from matplotlib) (4.39.2)
Requirement already satisfied: kiwisolver>=1.0.1 in ./.venv/lib64/python3.11/site-packages (from matplotlib) (1.4.4)
Requirement already satisfied: packaging>=20.0 in ./.venv/lib64/python3.11/site-packages (from matplotlib) (23.0)
Requirement already satisfied: pillow>=6.2.0 in ./.venv/lib64/python3.11/site-packages (from matplotlib) (9.4.0)
Requirement already satisfied: pyparsing>=2.3.1 in ./.venv/lib64/python3.11/site-packages (from matplotlib) (3.0.9)
Requirement already satisfied: python-dateutil>=2.7 in ./.venv/lib64/python3.11/site-packages (from matplotlib) (2.8.2)
Requirement already satisfied: pytz>=2020.1 in ./.venv/lib64/python3.11/site-packages (from pandas) (2023.2)
Requirement already satisfied: scipy>=1.3.2 in ./.venv/lib64/python3.11/site-packages (from scikit-learn) (1.10.1)
Requirement already satisfied: joblib>=1.1.1 in ./.venv/lib64/python3.11/site-packages (from scikit-learn) (1.2.0)
Requirement already satisfied: threadpoolctl>=2.0.0 in ./.venv/lib64/python3.11/site-packages (from scikit-learn) (3.1.0)
Requirement already satisfied: six>=1.5 in ./.venv/lib64/python3.11/site-packages (from python-dateutil>=2.7->matplotlib) (1.16.0)

[notice] A new release of pip available: 22.2.2 -> 23.0.1
[notice] To update, run: pip install --upgrade pip
Note: you may need to restart the kernel to use updated packages.
In [226]:
%matplotlib

import matplotlib.pyplot as plt
import numpy as np
import pandas as pd
import seaborn as sns
from sklearn import preprocessing

from ChernoffFace import *
import matplotlib.cm

from IPython.display import display, Markdown

plt.rcParams['figure.dpi'] = 300

Using matplotlib backend: agg
  1. Загружаем данные. Они представляют сведения об системе отопления и вентиляции (HVAC). Данные записывались в течение 14 дней, каждые 5 минут. Некоторые дни характеризуются аномальным поведением системы из-за неправильных настроек, которые привели к сбоям в работе системы. Задача: определить, в какие дни возникали аномальные ситуации в системе тепловентилирования и кондиционирования, какие параметры связаны с аномалиями.
  • Загрузим файл с исходными данными и выберем те столбцы, исследование которых предполагается вариантом, для удобства переименовав их.
In [227]:
df_full = pd.read_csv('./data/bldg-MC2.csv').astype({'Date/Time': 'datetime64'})

keys = filter(lambda x: 'F_3_Z_8' in x, df_full.keys())
df_orig = df_full[[*keys, "Date/Time"]]

def remove_id(key: str):
    if key == "Date/Time":
        return key
    
    return " ".join(key.split()[1:])

df_orig = df_orig.rename(columns=remove_id)

display(df_orig.info())
display(df_orig.describe())

<class 'pandas.core.frame.DataFrame'>
RangeIndex: 4032 entries, 0 to 4031
Data columns (total 11 columns):
 #   Column                           Non-Null Count  Dtype         
---  ------                           --------------  -----         
 0   Lights Power                     4032 non-null   float64       
 1   Equipment Power                  4032 non-null   float64       
 2   Thermostat Temp                  4032 non-null   float64       
 3   Thermostat Heating Setpoint      4032 non-null   float64       
 4   Thermostat Cooling Setpoint      4032 non-null   float64       
 5   VAV REHEAT Damper Position       4032 non-null   float64       
 6   REHEAT COIL Power                4032 non-null   float64       
 7   RETURN OUTLET CO2 Concentration  4032 non-null   float64       
 8   SUPPLY INLET Temperature         4032 non-null   float64       
 9   SUPPLY INLET Mass Flow Rate      4032 non-null   float64       
 10  Date/Time                        4032 non-null   datetime64[ns]
dtypes: datetime64[ns](1), float64(10)
memory usage: 346.6 KB

None
Lights Power Equipment Power Thermostat Temp Thermostat Heating Setpoint Thermostat Cooling Setpoint VAV REHEAT Damper Position REHEAT COIL Power RETURN OUTLET CO2 Concentration SUPPLY INLET Temperature SUPPLY INLET Mass Flow Rate
count 4032.000000 4032.000000 4032.000000 4032.000000 4032.000000 4032.000000 4032.000000 4032.000000 4032.000000 4032.000000
mean 382.896577 382.361052 24.265656 21.772634 24.875164 0.605345 1694.379812 597.874086 16.995249 0.301222
std 559.264492 313.188071 2.263242 4.224116 3.419606 0.357680 2180.469786 239.041545 5.796048 0.175148
min 0.000000 167.939000 18.324200 12.600000 15.560000 0.200000 0.000000 401.408000 12.280600 0.098800
25% 0.000000 167.939000 24.000100 21.000000 24.000000 0.200000 0.000000 421.000550 12.800200 0.098800
50% 0.000000 167.939000 24.001100 21.000000 24.000000 0.544850 0.000000 494.879500 12.801900 0.270600
75% 1199.564100 839.694800 24.001600 24.000000 25.875000 1.000000 4222.009550 686.973925 21.618925 0.493900
max 1199.564100 839.694800 32.224500 32.220000 35.200000 1.000000 12087.365100 1448.366100 40.041300 0.493900
  • Разделим данные на дни
In [228]:
dfs = [df for _, df in df_orig.groupby(df_orig["Date/Time"].dt.date)]

dfs = list(map(lambda df: df.drop("Date/Time", axis=1), dfs))

Ndays = len(dfs)

for i, df in enumerate(dfs):
    display(Markdown(f"День {i+1}"))
    display(df.describe())

День 1

Lights Power Equipment Power Thermostat Temp Thermostat Heating Setpoint Thermostat Cooling Setpoint VAV REHEAT Damper Position REHEAT COIL Power RETURN OUTLET CO2 Concentration SUPPLY INLET Temperature SUPPLY INLET Mass Flow Rate
count 288.000000 288.000000 288.000000 288.000000 288.000000 288.000000 288.0 288.000000 288.000000 288.000000
mean 503.983528 450.169735 23.511523 19.250000 24.895833 0.328044 0.0 560.934450 12.800678 0.162032
std 593.112652 332.143038 0.551798 2.435452 1.215399 0.159753 0.0 96.340623 0.010850 0.078895
min 0.000000 167.939000 22.501500 15.600000 24.000000 0.200000 0.0 446.366600 12.680400 0.098800
25% 0.000000 167.939000 23.005250 15.600000 24.000000 0.200000 0.0 462.861500 12.800000 0.098800
50% 0.000000 167.939000 23.848200 21.000000 24.000000 0.200000 0.0 545.981400 12.800700 0.098800
75% 1199.564100 839.694800 24.000800 21.000000 26.700000 0.477825 0.0 653.891600 12.801300 0.235975
max 1199.564100 839.694800 24.001500 21.000000 26.700000 0.646700 0.0 711.202400 12.891900 0.319400

День 2

Lights Power Equipment Power Thermostat Temp Thermostat Heating Setpoint Thermostat Cooling Setpoint VAV REHEAT Damper Position REHEAT COIL Power RETURN OUTLET CO2 Concentration SUPPLY INLET Temperature SUPPLY INLET Mass Flow Rate
count 288.000000 288.000000 288.000000 288.000000 288.000000 288.000000 288.0 288.000000 288.000000 288.000000
mean 508.148681 452.502221 23.570814 19.250000 24.895833 0.341358 0.0 572.221229 12.800780 0.168609
std 593.772701 332.512665 0.511852 2.435452 1.215399 0.173340 0.0 106.813916 0.011728 0.085606
min 0.000000 167.939000 22.715100 15.600000 24.000000 0.200000 0.0 447.323200 12.672700 0.098800
25% 0.000000 167.939000 22.959125 15.600000 24.000000 0.200000 0.0 464.209700 12.800000 0.098800
50% 0.000000 167.939000 23.975600 21.000000 24.000000 0.200000 0.0 555.156800 12.801100 0.098800
75% 1199.564100 839.694800 24.001400 21.000000 26.700000 0.494225 0.0 684.160150 12.801400 0.244150
max 1199.564100 839.694800 24.001800 21.000000 26.700000 0.669400 0.0 735.281300 12.887100 0.330600

День 3

Lights Power Equipment Power Thermostat Temp Thermostat Heating Setpoint Thermostat Cooling Setpoint VAV REHEAT Damper Position REHEAT COIL Power RETURN OUTLET CO2 Concentration SUPPLY INLET Temperature SUPPLY INLET Mass Flow Rate
count 288.000000 288.000000 288.000000 288.000000 288.000000 288.000000 288.0 288.000000 288.000000 288.000000
mean 495.653222 445.504765 23.724207 19.250000 24.895833 0.376494 0.0 570.342634 12.800657 0.185956
std 591.702086 331.353121 0.404597 2.435452 1.215399 0.219956 0.0 100.886557 0.012965 0.108629
min 0.000000 167.939000 22.916400 15.600000 24.000000 0.200000 0.0 452.602000 12.679700 0.098800
25% 0.000000 167.939000 23.301025 15.600000 24.000000 0.200000 0.0 470.378125 12.799100 0.098800
50% 0.000000 167.939000 24.000400 21.000000 24.000000 0.257700 0.0 550.226500 12.800400 0.127300
75% 1199.564100 839.694800 24.001200 21.000000 26.700000 0.489950 0.0 680.671250 12.801200 0.242000
max 1199.564100 839.694800 24.001900 21.000000 26.700000 0.917200 0.0 738.892500 12.913000 0.453000

День 4

Lights Power Equipment Power Thermostat Temp Thermostat Heating Setpoint Thermostat Cooling Setpoint VAV REHEAT Damper Position REHEAT COIL Power RETURN OUTLET CO2 Concentration SUPPLY INLET Temperature SUPPLY INLET Mass Flow Rate
count 288.000000 288.000000 288.000000 288.000000 288.000000 288.000000 288.000000 288.000000 288.000000 288.000000
mean 474.827456 433.842337 23.804253 20.420833 24.680208 0.553422 1083.012999 564.288872 14.954290 0.273343
std 587.642662 329.079844 0.329493 2.925340 1.110854 0.315791 1901.318151 86.462421 3.780952 0.155963
min 0.000000 167.939000 22.999500 15.600000 24.000000 0.200000 0.000000 461.059900 12.763100 0.098800
25% 0.000000 167.939000 23.675600 17.800000 24.000000 0.200000 0.000000 483.097900 12.800100 0.098800
50% 0.000000 167.939000 24.000500 21.000000 24.000000 0.499000 0.000000 540.725750 12.800500 0.246400
75% 1199.564100 839.694800 24.001025 21.000000 25.600000 0.823425 0.000000 645.305675 12.865975 0.406750
max 1199.564100 839.694800 24.001400 24.000000 26.700000 1.000000 4710.442400 716.496100 22.166500 0.493900

День 5

Lights Power Equipment Power Thermostat Temp Thermostat Heating Setpoint Thermostat Cooling Setpoint VAV REHEAT Damper Position REHEAT COIL Power RETURN OUTLET CO2 Concentration SUPPLY INLET Temperature SUPPLY INLET Mass Flow Rate
count 288.0 288.000 288.000000 288.0 288.0 288.0 288.000000 288.000000 288.000000 288.0000
mean 0.0 167.939 24.000690 24.0 24.0 1.0 4247.139773 433.469634 21.244178 0.4939
std 0.0 0.000 0.000478 0.0 0.0 0.0 699.705141 30.700542 1.391843 0.0000
min 0.0 167.939 23.999700 24.0 24.0 1.0 2534.397800 403.713100 17.837700 0.4939
25% 0.0 167.939 24.000200 24.0 24.0 1.0 4067.505775 405.727650 20.887025 0.4939
50% 0.0 167.939 24.000650 24.0 24.0 1.0 4469.623800 419.045800 21.687150 0.4939
75% 0.0 167.939 24.001200 24.0 24.0 1.0 4788.098400 462.267900 22.320975 0.4939
max 0.0 167.939 24.001400 24.0 24.0 1.0 4967.211000 496.143500 22.677400 0.4939

День 6

Lights Power Equipment Power Thermostat Temp Thermostat Heating Setpoint Thermostat Cooling Setpoint VAV REHEAT Damper Position REHEAT COIL Power RETURN OUTLET CO2 Concentration SUPPLY INLET Temperature SUPPLY INLET Mass Flow Rate
count 288.0 288.000 288.000000 288.0 288.0 288.0 288.000000 288.000000 288.000000 288.0000
mean 0.0 167.939 24.001470 24.0 24.0 1.0 4475.198956 403.850090 21.698237 0.4939
std 0.0 0.000 0.000221 0.0 0.0 0.0 428.359137 1.988824 0.852114 0.0000
min 0.0 167.939 24.001000 24.0 24.0 1.0 3553.450100 401.408000 19.864900 0.4939
25% 0.0 167.939 24.001300 24.0 24.0 1.0 4031.361150 402.108600 20.815300 0.4939
50% 0.0 167.939 24.001500 24.0 24.0 1.0 4661.311650 403.179600 22.068550 0.4939
75% 0.0 167.939 24.001700 24.0 24.0 1.0 4838.499475 405.509325 22.420900 0.4939
max 0.0 167.939 24.001800 24.0 24.0 1.0 5000.825600 407.910700 22.744000 0.4939

День 7

Lights Power Equipment Power Thermostat Temp Thermostat Heating Setpoint Thermostat Cooling Setpoint VAV REHEAT Damper Position REHEAT COIL Power RETURN OUTLET CO2 Concentration SUPPLY INLET Temperature SUPPLY INLET Mass Flow Rate
count 288.000000 288.000000 288.000000 288.000000 288.000000 288.000000 288.000000 288.000000 288.000000 288.000000
mean 524.809294 461.832163 23.837475 21.029167 24.323958 0.522375 1099.516296 548.899967 14.985247 0.258007
std 596.113624 333.823582 0.402874 2.172295 0.792308 0.298195 2125.588032 125.727824 4.227184 0.147273
min 0.000000 167.939000 22.581000 15.600000 24.000000 0.200000 0.000000 402.578600 12.447800 0.098800
25% 0.000000 167.939000 24.000300 21.000000 24.000000 0.205325 0.000000 403.828850 12.799900 0.101425
50% 0.000000 167.939000 24.001300 21.000000 24.000000 0.473550 0.000000 537.616350 12.801000 0.233900
75% 1199.564100 839.694800 24.001600 21.000000 24.000000 0.706400 0.000000 665.107150 12.805025 0.348900
max 1199.564100 839.694800 24.001900 24.000000 26.700000 1.000000 5415.237300 760.532500 23.567900 0.493900

День 8

Lights Power Equipment Power Thermostat Temp Thermostat Heating Setpoint Thermostat Cooling Setpoint VAV REHEAT Damper Position REHEAT COIL Power RETURN OUTLET CO2 Concentration SUPPLY INLET Temperature SUPPLY INLET Mass Flow Rate
count 288.000000 288.000000 288.000000 288.000000 288.000000 288.000000 288.000000 288.000000 288.000000 288.000000
mean 524.809294 461.832163 24.509718 21.404167 24.436181 0.497305 232.659550 933.288262 15.558582 0.247842
std 596.113624 333.823582 4.163353 6.812276 6.838784 0.387182 926.294824 367.133018 5.605668 0.191152
min 0.000000 167.939000 18.324200 12.600000 15.560000 0.200000 0.000000 456.966500 12.280600 0.098800
25% 0.000000 167.939000 19.485175 12.600000 15.560000 0.200000 0.000000 480.137175 12.799275 0.098800
50% 0.000000 167.939000 26.670700 26.670000 29.700000 0.200000 0.000000 1008.062900 12.801800 0.098800
75% 1199.564100 839.694800 27.656900 26.670000 29.700000 1.000000 112.354475 1278.395650 15.685775 0.493900
max 1199.564100 839.694800 30.465500 26.670000 29.700000 1.000000 12087.365100 1424.687400 39.986800 0.493900

День 9

Lights Power Equipment Power Thermostat Temp Thermostat Heating Setpoint Thermostat Cooling Setpoint VAV REHEAT Damper Position REHEAT COIL Power RETURN OUTLET CO2 Concentration SUPPLY INLET Temperature SUPPLY INLET Mass Flow Rate
count 288.000000 288.000000 288.000000 288.000000 288.000000 288.000000 288.000000 288.000000 288.000000 288.000000
mean 516.478988 457.167192 24.621200 21.393750 24.397500 0.500893 212.522714 1098.229528 14.674794 0.249204
std 595.002834 333.201539 4.128058 6.823466 6.857414 0.387310 876.449985 243.391339 4.546878 0.191238
min 0.000000 167.939000 18.599300 12.600000 15.560000 0.200000 0.000000 753.613000 12.352700 0.098800
25% 0.000000 167.939000 19.717550 12.600000 15.560000 0.200000 0.000000 867.380150 12.799700 0.098800
50% 0.000000 167.939000 26.670450 26.670000 29.700000 0.200000 0.000000 1072.915200 12.801300 0.098800
75% 1199.564100 839.694800 27.937000 26.670000 29.700000 1.000000 26.056275 1357.545275 13.756875 0.493900
max 1199.564100 839.694800 29.703800 26.670000 29.700000 1.000000 12087.365100 1448.366100 39.986700 0.493900

День 10

Lights Power Equipment Power Thermostat Temp Thermostat Heating Setpoint Thermostat Cooling Setpoint VAV REHEAT Damper Position REHEAT COIL Power RETURN OUTLET CO2 Concentration SUPPLY INLET Temperature SUPPLY INLET Mass Flow Rate
count 288.000000 288.000000 288.000000 288.000000 288.000000 288.000000 288.000000 288.000000 288.000000 288.000000
mean 441.506231 415.182454 28.303377 25.368958 30.722778 0.232185 1923.220353 727.312097 24.442192 0.141968
std 579.528628 324.535985 4.192170 7.496090 4.878082 0.101725 2498.883820 113.396914 11.939473 0.070476
min 0.000000 167.939000 20.094400 12.600000 15.560000 0.200000 0.000000 482.408700 12.487500 0.098800
25% 0.000000 167.939000 23.866725 15.600000 26.700000 0.200000 0.000000 716.217100 12.800200 0.098800
50% 0.000000 167.939000 32.221100 32.220000 35.200000 0.200000 829.817850 756.009350 21.045550 0.098800
75% 1199.564100 839.694800 32.221900 32.220000 35.200000 0.200000 2644.896675 783.376075 38.813800 0.188225
max 1199.564100 839.694800 32.224500 32.220000 35.200000 1.000000 12087.365100 973.614200 40.041300 0.493900

День 11

Lights Power Equipment Power Thermostat Temp Thermostat Heating Setpoint Thermostat Cooling Setpoint VAV REHEAT Damper Position REHEAT COIL Power RETURN OUTLET CO2 Concentration SUPPLY INLET Temperature SUPPLY INLET Mass Flow Rate
count 288.000000 288.000000 288.000000 288.000000 288.000000 288.000000 288.000000 288.000000 288.000000 288.000000
mean 499.818375 447.837250 23.950311 20.420833 24.680208 0.587012 1026.515424 568.351603 14.842355 0.289932
std 592.422482 331.756543 0.244454 2.925340 1.110854 0.316005 1806.219462 98.776765 3.591064 0.156068
min 0.000000 167.939000 23.371700 15.600000 24.000000 0.200000 0.000000 450.978900 12.762200 0.098800
25% 0.000000 167.939000 24.000300 17.800000 24.000000 0.200000 0.000000 470.843675 12.800400 0.098800
50% 0.000000 167.939000 24.001000 21.000000 24.000000 0.543300 0.000000 544.919150 12.800900 0.268350
75% 1199.564100 839.694800 24.001500 21.000000 25.600000 0.919800 0.000000 660.461250 12.876875 0.454300
max 1199.564100 839.694800 24.870300 24.000000 26.700000 1.000000 4582.563900 745.095800 21.911900 0.493900

День 12

Lights Power Equipment Power Thermostat Temp Thermostat Heating Setpoint Thermostat Cooling Setpoint VAV REHEAT Damper Position REHEAT COIL Power RETURN OUTLET CO2 Concentration SUPPLY INLET Temperature SUPPLY INLET Mass Flow Rate
count 288.000000 288.000000 288.000000 288.0 288.0 288.0 288.000000 288.000000 288.000000 288.0000
mean 199.927350 279.898300 24.000808 24.0 24.0 1.0 4137.211673 439.857060 21.025352 0.4939
std 447.829303 250.784374 0.000524 0.0 0.0 0.0 785.924157 27.783050 1.563816 0.0000
min 0.000000 167.939000 23.999800 24.0 24.0 1.0 2587.249100 407.411600 17.941900 0.4939
25% 0.000000 167.939000 24.000300 24.0 24.0 1.0 3406.121425 411.430700 19.569850 0.4939
50% 0.000000 167.939000 24.000850 24.0 24.0 1.0 4558.965400 441.333000 21.864850 0.4939
75% 0.000000 167.939000 24.001300 24.0 24.0 1.0 4782.725000 462.379125 22.310100 0.4939
max 1199.564100 839.694800 24.001500 24.0 24.0 1.0 4957.034600 496.260600 22.657000 0.4939

День 13

Lights Power Equipment Power Thermostat Temp Thermostat Heating Setpoint Thermostat Cooling Setpoint VAV REHEAT Damper Position REHEAT COIL Power RETURN OUTLET CO2 Concentration SUPPLY INLET Temperature SUPPLY INLET Mass Flow Rate
count 288.000000 288.000000 288.000000 288.0 288.0 288.0 288.000000 288.000000 288.000000 288.0000
mean 199.927350 279.898300 24.001308 24.0 24.0 1.0 4215.589911 410.147666 21.181914 0.4939
std 447.829303 250.784374 0.000302 0.0 0.0 0.0 820.904511 5.385331 1.632757 0.0000
min 0.000000 167.939000 24.000800 24.0 24.0 1.0 2725.586600 404.058100 18.218600 0.4939
25% 0.000000 167.939000 24.001000 24.0 24.0 1.0 3406.092350 406.277050 19.571550 0.4939
50% 0.000000 167.939000 24.001400 24.0 24.0 1.0 4609.780300 408.283100 21.965950 0.4939
75% 0.000000 167.939000 24.001500 24.0 24.0 1.0 4934.070525 412.638400 22.611025 0.4939
max 1199.564100 839.694800 24.001700 24.0 24.0 1.0 5106.302300 425.067400 22.953600 0.4939

День 14

Lights Power Equipment Power Thermostat Temp Thermostat Heating Setpoint Thermostat Cooling Setpoint VAV REHEAT Damper Position REHEAT COIL Power RETURN OUTLET CO2 Concentration SUPPLY INLET Temperature SUPPLY INLET Mass Flow Rate
count 288.000000 288.000000 288.000000 288.000000 288.000000 288.000000 288.000000 288.000000 288.000000 288.000000
mean 470.662303 431.509852 23.882030 21.029167 24.323958 0.535745 1068.729723 539.044120 14.924235 0.264610
std 586.738402 328.573458 0.334942 2.172295 0.792308 0.303520 2065.732748 114.400789 4.108075 0.149906
min 0.000000 167.939000 22.792800 15.600000 24.000000 0.200000 0.000000 403.810200 12.559100 0.098800
25% 0.000000 167.939000 24.000400 21.000000 24.000000 0.225650 0.000000 405.584825 12.799900 0.111425
50% 0.000000 167.939000 24.001150 21.000000 24.000000 0.479700 0.000000 532.604900 12.800600 0.236900
75% 1199.564100 839.694800 24.001600 21.000000 24.000000 0.827650 0.000000 660.301700 12.803125 0.408825
max 1199.564100 839.694800 24.034500 24.000000 26.700000 1.000000 5216.155800 706.925900 23.172000 0.493900
  • Соберём в таблицу ежедневные средние значения по каждому дню и изобразим их на графике.
In [235]:
df_means = pd.DataFrame([df.mean() for df in dfs], columns=dfs[0].keys())

display(df_means)

fig, ax = plt.subplots(df_means.shape[1], sharex=True)

for i, key in enumerate(df_means):
    ax[i].plot(df_means[key])
    ax[i].set_title(key)
    ax[i].set_ylabel("Величина")
    ax[i].grid()

_ = ax[-1].set_xticks(range(Ndays))
_ = ax[-1].set_xticklabels(map(str, range(1, 15)))
_ = ax[-1].set_xlabel("День")


fig.set_size_inches(10, 13)
fig.tight_layout()

display(fig)
Lights Power Equipment Power Thermostat Temp Thermostat Heating Setpoint Thermostat Cooling Setpoint VAV REHEAT Damper Position REHEAT COIL Power RETURN OUTLET CO2 Concentration SUPPLY INLET Temperature SUPPLY INLET Mass Flow Rate
0 0.420139 0.420139 0.673349 0.675926 0.331790 0.286644 0.000000 0.432600 0.568693 0.286638
1 0.423611 0.423611 0.665046 0.675926 0.331790 0.301146 0.000000 0.433737 0.597386 0.301159
2 0.413194 0.413194 0.744180 0.675926 0.331790 0.246088 0.000000 0.411263 0.518459 0.246064
3 0.395833 0.395833 0.803227 0.573909 0.251929 0.441778 0.229917 0.404128 0.233021 0.441770
4 0.000000 0.000000 0.582108 0.000000 0.000000 0.000000 0.704017 0.321934 0.703861 0.000000
5 0.000000 0.000000 0.587674 0.000000 0.000000 0.000000 0.636842 0.375550 0.636774 0.000000
6 0.437500 0.437500 0.884281 0.646329 0.119985 0.402969 0.203041 0.408772 0.228186 0.402954
7 0.437500 0.437500 0.509461 0.625740 0.627736 0.371631 0.019248 0.492210 0.118312 0.377225
8 0.430556 0.430556 0.542294 0.625000 0.625000 0.376116 0.017582 0.496027 0.084030 0.380673
9 0.368056 0.368056 0.676744 0.650813 0.772036 0.040231 0.159110 0.498576 0.433867 0.109259
10 0.416667 0.416667 0.386101 0.573909 0.251929 0.483765 0.224005 0.399068 0.227347 0.483757
11 0.166667 0.166667 0.593137 0.000000 0.000000 0.000000 0.654052 0.365175 0.653953 0.000000
12 0.166667 0.166667 0.564429 0.000000 0.000000 0.000000 0.625864 0.289851 0.625832 0.000000
13 0.392361 0.392361 0.877209 0.646329 0.119985 0.419682 0.204888 0.446146 0.222855 0.419667
No description has been provided for this image

Из графиков видно, что Lights Power и Equipment Power, а так же VAV REHEAT Damper Position и SUPPLY INLET Mass Flow Rate представляют примерно одни и те же зависимости с точностью до масштабного коэффициента, поэтому можно отбросить их при анализе.

  • Нормируем данные в каждой колонке
In [230]:
scaled_dfs = dfs
for i, df in enumerate(dfs):
    scaled_tmp = pd.DataFrame(
        preprocessing.MinMaxScaler().fit_transform(df), 
        columns=df.columns
    )
    
    scaled_dfs[i]=scaled_tmp

dfs = scaled_dfs
  • Построим матрицы корреляции для каждого дня
In [231]:
dfs_corr = [df.corr() for df in dfs]

for i, df_corr in enumerate(dfs_corr):
    fig, ax = plt.subplots()

    display(Markdown(f"День {i+1}"))

    mask = np.triu(np.ones_like(df_corr))

    _ = sns.heatmap(
            df_corr,
            cbar=False,
            ax=ax,
            annot=True,
            cmap='coolwarm',
            mask=mask,
            vmin=-1,
            vmax=1,
        )

    display(fig)

День 1

No description has been provided for this image

День 2

No description has been provided for this image

День 3

No description has been provided for this image

День 4

No description has been provided for this image

День 5

No description has been provided for this image

День 6

No description has been provided for this image

День 7

No description has been provided for this image

День 8

No description has been provided for this image

День 9

No description has been provided for this image

День 10

No description has been provided for this image

День 11

No description has been provided for this image

День 12

No description has been provided for this image

День 13

No description has been provided for this image

День 14

No description has been provided for this image
  • Выберем те ячейки матрицы корреляции, которые в достаточной степени меняются в зависимости от дня:
In [232]:
corr_matrix_size = dfs_corr[0].shape
useful = np.full(corr_matrix_size, False)
useful_l = np.ndarray((0, Ndays))

# for i in range(len(dfs_corr)):
#     dfs_corr[i] = dfs_corr[i].fillna(0)

import itertools
from functools import reduce

def norm(x):
    return (x-np.min(x))/(np.max(x)-np.min(x))

for pos in itertools.combinations_with_replacement(range(10), 2):
    elms = [df_corr.iat[pos] for df_corr in dfs_corr]

    mean = np.mean(elms)

    # print(elms)

    useful[pos] = reduce(lambda res, el: res or (abs(el - mean) > 0.1), elms, False)

    if useful[pos]:
        useful_l = np.append(useful_l, [norm(elms)], axis=0)
        print(dfs_corr[0].keys()[pos[0]], dfs_corr[0].index[pos[1]], sep=', ')

Thermostat Temp, RETURN OUTLET CO2 Concentration
Thermostat Temp, SUPPLY INLET Temperature
RETURN OUTLET CO2 Concentration, SUPPLY INLET Temperature

Наконец, построим лица Чернова для каждого дня.

In [233]:
f = plt.figure(dpi=300, figsize=(14, 6))

with_regular = np.array([*useful_l.T, np.full(len(useful_l.T[0]), 0.5)])

fig = chernoff_face(
    data=with_regular,
    color_mapper=matplotlib.cm.Pastel1,
    n_columns=7,
    long_face=False,
    n_rows=3,
    figure=f,
    titles=[*[f"День {i+1}" for i in range(Ndays)], "Референс"]
)

display(fig)
No description has been provided for this image

Таблица. Анализ данных с помощью лиц Чернова

Вопрос Ответ
1. Определение параметров для визуализации с помощью лиц Чернова. Втечение 14-ти дней каждые 5 минут HVAC оборудование измеряло следующие величины:
Lights Power
Equipment Power
Thermostat Temp
Thermostat Heating Setpoint
Thermostat Cooling Setpoint
VAV REHEAT Damper Position
REHEAT COIL Power
RETURN OUTLET CO2 Concentration
SUPPLY INLET Temperature
SUPPLY INLET Mass Flow Rate
(названия соответствуют столбцам исходных данных).
2. Схема визуализации Чернова. Представлена на последнем рисунке. Для построения используются величины, характеризующие корреляцию между температурой термостата, возвращаемой концентрации CO2 и температуры поступающего воздуха, которые кодируются цветом, высотой головы и размером глаз. Так же ниже приведён вариант, соответствующий норме.
3. Определение, в какие дни возникали аномальные ситуации в системе тепловентилирования и кондиционирования. Как видно из графиков для средних значений и лиц Чернова, в 5 и 6, 9 и 10, и 12 дни.
4. Какие параметры связаны с аномалиями. Аномалии наблюдаются в отключении электичества для оборудования в 5 и 6 дни. Также о проблемах сигнализируют нарушения корреляции для температуры термостата и возвращаемой концентрации CO2 и температуры поступающего воздуха.
5. Что могло послужит причиной аномалии? (дополнительный необязательный вопрос) Судя по всему, причиной аномалии служат перебои в электропитании.