There is nothing like upgrading minor versions of software to find they don’t work as expected. But nonetheless this is the typical case; especially when you start fiddling with core code without submitting PR’s. Today to improve some of my code base I upgraded OMD from 1.20 to 1.30 to enable the WATO webapi.py API.
So you can understand the full struggle here is the full bash history.
1001 uptime
1002 updatedb
1003 locate webapi.py
1004 locate check_mk
1005 locate check_mk | grep -v doc
1006 omd versions
1007 su - core
1008 rpm -qa omd
1009 uname -a
1010 cd
1011 wget http://files.omdistro.org/releases/1.30/omd-1.30.rhel6.x86_64.rpm
1012 ll
1013 cd /opt/
1014 ls
1015 ll
1016 du -sh omd/
1017 clear
1018 ll
1019 tar cvf omd-backup.tar omd/
1020 ll
1021 ls -lah
1022 uptime
1023 df -h
1024 locate inventory.py
1025 uptime
1026 w
1027 locate wato
1028 ll
1029 ls -lrt
1030 cd
1031 ls
1032 ll
1033 rpm -U omd-1.30.rhel6.x86_64.rpm
1034 yum search freeradius
1035 yum -y install freeradius-utils
1036 uptime
1037 yum update
1038 rpm -U omd-1.30.rhel6.x86_64.rpm
1039 su - core
1040 updatedb
1041 locate wato
1042 locate webapi
1043 cat /opt/omd/versions/1.30/share/check_mk/web/htdocs/webapi.py
1044 clear
1045 cd /etc/httpd/
1046 ls
1047 ll
1048 cd conf.d/
1049 ls
1050 ll
1051 cat fcgid.conf
1052 cat php.conf
1053 clear
1054 ls
1055 cat zzz_omd.conf
1056 cd /omd/apache/
1057 ls
1058 ll
1059 vim core.conf
1060 cat /omd/sites/core/etc/apache/mode.conf
1061 locate \*conf
1062 locate \*conf | grep omd
1063 su - co
1064 locate \*conf | grep omd
1065 cd /opt/omd/versions/1.30/skel/etc/apache/
1066 ls
1067 ll
1068 cat apache
1069 cat apache.conf
1070 ll
1071 cd conf.d
1072 ls
1073 pwd
1074 ll
1075 cat 01_python.conf
1076 cat 02.
1077 cat omd.conf
1078 ll
1079 cat site.conf
1080 cp omd.conf /etc/httpd/conf.d/
1081 vim /etc/httpd/conf.d/omd.conf
1082 locate htdocs
1083 vim /etc/httpd/conf.d/omd.conf
1084 locate htdocs | grep omd | grep share
1085 vim /etc/httpd/conf.d/omd.conf
1086 locate htdocs | grep 'share/omd'
1087 vim /etc/httpd/conf.d/omd.conf
1088 service httpd status
1089 service httpd restart
1090 locate webapi.py
1091 cd /opt/omd/versions/1.30/share/check_mk/
1092 ls
1093 ll
1094 find | grep conf
1095 pwd
1096 cd /etc/httpd/conf.d/
1097 ls
1098 ll
1099 cat zzz_omd.conf
1100 ls /omd/apache/
1101 locate conf | grep omd
1102 ls /omd/apache/
1103 ls -lah /
1104 ls -lah /omd/omd
1105 ls -lah /opt/omd
1106 ls -lah /opt/omd/apache/
1107 date
1108 cat /opt/omd/apache/core.conf
1109 cat /omd/sites/core/etc/apache/mode.conf
1110 cd /omd/sites/core/
1111 ls
1112 ll
1113 su - core
1114 yum -y install libdbi libdbi-devel
1115 su - core
1116 updatedb
1117 service omd restart
1118 ll
1119 pwd
1120 cd ..
1121 ls
1122 ll
1123 cd ..
1124 ls
1125 ll
1126 uptime
1127 find -mmin -5
1128 find -mmin -5 | grep -v perf
1129 find -mmin -5 | grep -v perf | grep -v cache
1130 find -mmin -5 | grep -v perf | grep -v cache | grep -v counters
1131 tail ./sites/core/var/log/apache/error_log
1132 omd status
1133 updatedb
1134 find / -mmin -1 -type f
1135 find / -mmin -1 -type f | grep -v proc | grep -v sys
1136 tail /var/log/audit/audit.log
1137 tail /var/log/httpd/error_log
1138 su - core
1139 omd create testing
1140 omd start testing
1141 find /etc/httpd/ -mmin -1
1142 find / -mmin -1 | grep -v proc | grep -v sys
1143 find / -mmin -1 | grep -v proc | grep -v sys | grep- v
1144 cd /opt/omd/apache/
1145 ls
1146 ll
1147 cat testing.conf
1148 cat /omd/sites/testing/etc/apache/mode.conf
1149 ll
1150 cd /omd/sites/core/etc/apache/
1151 cp mode.conf backup-mode.conf-broken
1152 pwd
1153 ls
1154 cd -
1155 locate core.conf
1156 cd /opt/omd/apache/
1157 ll
1158 cat core.conf
1159 cd /omd/sites/core/etc/apache/
1160 ls
1161 ls -lah
1162 cat apache-own.conf
1163 vim apache-own.conf
1164 ll
1165 cp apache-own.conf apache-own.conf.old
1166 rm -vf apache-own.conf
1167 pwd
1168 cp /omd/sites/testing/etc/apache/apache-own.conf .
1169 vim apache-own.conf
1170 service httpd restart
1171 history
Now that you see how much ls and cd are used in my general terminal day to day lets go through the quick and easy way to do this update.
Backup
As with any decent systems administrator I made a backup first so if anything did not occur as expected and I couldn’t fix it in a reasonable timeframe I could easily revert. Use your preferred solution.
Download RPM and upgrade
This will just download the package and install it; you need to actually upgrade via OMD to make any changes.
wget http://files.omdistro.org/releases/1.30/omd-1.30.rhel6.x86_64.rpm
rpm -U omd-1.30.rhel6.x86_64.rpm
Upgrade via the site user
First, su to the user you have configured for OMD and run:
omd stop
omd update
omd start
Troubleshooting apache errors
At this point; you have dotted your i’s and crossed your fingers and hoped that the maintainers of OMD would have handled all issues. But you come across a “OMD: Site Not Started” error when you access via HTTP.
The problem? Apache isn’t configured to run the new code. Just take a look at my full “root” user history above.
So if you ever come across a problem you can’t solve; pull this trick; it comes from troubleshooting 101; REPLICATE THE PROBLEM. I try this by creating a new OMD Site which then works out of the box. With this it is a matter of backing up old configuration files and in vim mass replacing with %s//g.