# Troubleshooting
# Automatic system repair
Use kctl repair
to automatically detect and fix most common issues on the server.
- Open terminal.
- Run:
kctl repair
It shows "kctl: command not found" error.
That means you must upgrade the server configuration first.
# Give temporary access for customer support
- Open terminal.
- Run:
kctl support-team-access allow
Access is valid for 5 days. Disable:
kctl support-team-access deny
It shows "kctl: command not found" error.
That means you must upgrade the server configuration first.
# FAQ
The server works slow, 502 errors.
Open Maintenance — Status in Keitaro. Check the value in the line CPU Stolen.
If value greater than 0, that means the cloud server doesn't receive enough CPU time. You must report about that issue to the support of the cloud provider.
Not enough space on the server.
If server has 10%-15% space, try Optimize the database, on the other case, increase the server size or delete all the statistics.
I increased RAM, but Keitaro does not see it.
You must upgrade server configuration.
How to understand that everything works well?
Keitaro shows yellow exclamation mark at the top of a screen if something is wrong. You may also check System Log.
Errors "Some internal error" or "See system log".
Open Maintenance → Logs → System log
.
"RedisException. OOM command not allowed when used memory > maxmemory" error.
- Go to Maintenance → Settings → System
- Set a lower LP Token expire time.
Set lower uniqueness expire time in the campaigns.
Upgrade the server configuration.
Keitaro cannot refresh the license.
- If you changed
/etc/resolve.conf
, revert the changes. - If you changed firewall settings, check if it allows outgoing connections.
Got error 28 from the storage engine.
It means not enough disk space
.
To show disc space in a server console, use the following command:
df -h
Redis is LOADING the dataset.
That solution deletes all the unprocessed statistics, but fixes the issue fast.
Run in the server terminal:
rm -rf /var/lib/redis/*
systemctl restart redis
2
Database is corrupted.
Mostly, this occurs when MySQL database of your tracker crashed because of major failure in SSD read or writing error (bad blocks, IO error, bad or defective SSD drive).
- In the terminal, run:
sudo touch /forcefsck
- Reboot the server:
reboot
- Start MySQL DB service in normal mode
systemctl start mysql
How to free inodes usage?
If your tracker started to work incorrectly, and you see messages about low disk space (while df -h
showing you
that you have enough disk space), it is possible that inodes usage on your server is close to its capacity.
Inodes — are file descriptors in Linux, which store the attributes and disk block locations of the object's data.
File—system object attributes may include metadata (times of last change, access, modification), as well as the owner and
permission data. When the number of inodes is close to 0 - that means your OS is insufficient of file descriptors (
inodes).
In such a case, you need to delete those files. For searching inodes, you need to use the following command:
df -i
We will receive the output of the structure of your file system with a list of inodes number directly. Now you need to follow the directory with the biggest number of inodes in your system — you need to go to the directory with the biggest inodes number and look for the folder with the biggest inodes number. Finally, you will find the folder with the biggest inode count number. After that, you need to delete this folder with the following command:
rm -rf deletedfolder/
Where deletedfolder/
is the name of the folder with the biggest inode number which you need to delete.
WARNING Please verify the folder, which you are going to delete — your target is to delete the folder with most of the file descriptors (or biggest inode number), but NOT the system files of your OS or a server. If you are not sure what are you doing, it is better to contact with technical support, which will help you gladly to free inodes usage on your server. In other cases, you are risking to make your server inoperable.
Server works slowly.
This may happen because of the lack of CPU resources for your virtual server. Run the command:
vmstat 1 5
As a result, you'll see the following table:
procs -----------memory---------- ---swap-- -----io---- -system-- ------cpu-----
r b swpd free buff cache si so bi bo in cs us sy id wa st
111 0 0 3438984 20280 603200 0 0 11 20 38 85 10 2 32 0 56
139 0 0 3435676 20280 603220 0 0 0 0 936 318 15 3 0 0 82
126 0 0 3434312 20280 603212 0 0 0 0 903 198 14 4 0 0 82
92 0 0 3450964 20280 603868 0 0 0 0 885 198 13 3 0 0 83
145 1 0 3452128 20280 604412 0 0 448 0 993 254 14 2 0 0 84
2
3
4
5
6
7
Pay attention to the last column st
. If the value is more than 0, your VPS has insufficient resources. Contact the server provider in this case.
How do I check if my server has HDD or SSD?
Check the disks' names here:
cat /proc/scsi/scsi
Error "malformed worker response".
- Open SSH terminal.
- Run the following command:
cd /var/www/keitaro/application
find . -type f -name "*.php" -exec sed -i 's/?>//g' {} +
2
# Optimizing database
Open the Maintenance — Status section in Keitaro. If there is a Clickhouse upgrade
notification, contact support for the transition to Clickhouse.
If the Maintenance — Status looks like this, Clickhouse is installed, and optimization is not required.
# Deleting all the statistics
kctl run cli-php system:delete_mysql_stats
The command will ask for confirmation to remove all statistics from MYSQL:
[WARNING] Important! The following will happen:
- old versions tables removal,
- clicks stats to the exact date removal (if specified in parameters),
- optimization of the clicks in the stats,
- cleaning all the current stats completely (if specified in parameters).
These inquiries will block the database
during the implementation and require additional free space on a disk.
2
3
4
5
6
7
8
Type in yes
and press ENTER
to confirm.
Automatic cleaning.
To turn on automatic data cleaning, set Stats store term in the settings.