EN RU
Pricing Blog

# Troubleshooting

# Automatic system repair

Use kctl rescue to automatically detect and fix most common issues on the server.

  1. Open terminal.
  2. Run:
kctl rescue
1
It shows "kctl: command not found" error.

That means you must upgrade the server configuration first.

# Give temporary access for customer support

  1. Open terminal.
  2. Run:
kctl support-team-access allow
1

Access is valid for 5 days. Disable:

kctl support-team-access deny
1
It shows "kctl: command not found" error.

That means you must upgrade the server configuration first.

# FAQ

The server works slow, 502 errors.

Open Maintenance — Status in Keitaro. Check the value in the line CPU Stolen.

If value greater than 0, that means the cloud server doesn't receive enough CPU time. You must report about that issue to the support of the cloud provider.

Not enough space on the server.

If server has 10%-15% space, try Optimize the database, on the other case, increase the server size or delete all the statistics.

I increased RAM, but Keitaro does not see it.

You must upgrade server configuration.

How to understand that everything works well?

Keitaro shows yellow exclamation mark at the top of a screen if something is wrong. You may also check System Log.

Errors "Some internal error" or "See system log".

Open Maintenance → Logs → System log.

"RedisException. OOM command not allowed when used memory > maxmemory" error.
  1. Go to Maintenance → Settings → System
  • Set a lower LP Token expire time.

  1. Set lower uniqueness expire time in the campaigns.

  2. Upgrade the server configuration.

Keitaro cannot refresh the license.
  • If you changed /etc/resolve.conf, revert the changes.
  • If you changed firewall settings, check if it allows outgoing connections.
Got error 28 from the storage engine.

It means not enough disk space.

To show disc space in a server console, use the following command:

df -h
1
Redis is LOADING the dataset.

That solution deletes all the unprocessed statistics, but fixes the issue fast.

Run in the server terminal:

rm -rf /var/lib/redis/*
systemctl restart redis
1
2
Database is corrupted.

Mostly, this occurs when MySQL database of your tracker crashed because of major failure in SSD read or writing error (bad blocks, IO error, bad or defective SSD drive).

  1. In the terminal, run:

sudo touch /forcefsck
1
  1. Reboot the server:

reboot
1
  1. Start MySQL DB service in normal mode

systemctl start mysql
1
How to free inodes usage?

If your tracker started to work incorrectly, and you see messages about low disk space (while df -h showing you that you have enough disk space), it is possible that inodes usage on your server is close to its capacity. Inodes — are file descriptors in Linux, which store the attributes and disk block locations of the object's data. File—system object attributes may include metadata (times of last change, access, modification), as well as the owner and permission data. When the number of inodes is close to 0 - that means your OS is insufficient of file descriptors ( inodes).

In such a case, you need to delete those files. For searching inodes, you need to use the following command:

df -i
1

We will receive the output of the structure of your file system with a list of inodes number directly. Now you need to follow the directory with the biggest number of inodes in your system — you need to go to the directory with the biggest inodes number and look for the folder with the biggest inodes number. Finally, you will find the folder with the biggest inode count number. After that, you need to delete this folder with the following command:

rm -rf deletedfolder/
1

Where deletedfolder/ is the name of the folder with the biggest inode number which you need to delete.

WARNING Please verify the folder, which you are going to delete — your target is to delete the folder with most of the file descriptors (or biggest inode number), but NOT the system files of your OS or a server. If you are not sure what are you doing, it is better to contact with technical support, which will help you gladly to free inodes usage on your server. In other cases, you are risking to make your server inoperable.

Server works slowly.

This may happen because of the lack of CPU resources for your virtual server. Run the command:

vmstat 1 5
1

As a result, you'll see the following table:

procs -----------memory---------- ---swap-- -----io---- -system-- ------cpu-----
 r  b   swpd   free   buff  cache   si   so    bi    bo   in   cs us sy id wa st
111  0      0 3438984  20280 603200    0    0    11    20   38   85 10  2 32  0 56
139  0      0 3435676  20280 603220    0    0     0     0  936  318 15  3  0  0 82
126  0      0 3434312  20280 603212    0    0     0     0  903  198 14  4  0  0 82
92   0      0 3450964  20280 603868    0    0     0     0  885  198 13  3  0  0 83
145  1      0 3452128  20280 604412    0    0   448     0  993  254 14  2  0  0 84
1
2
3
4
5
6
7

Pay attention to the last column st. If the value is more than 0, your VPS has insufficient resources. Contact the server provider in this case.

How do I check if my server has HDD or SSD?

Check the disks' names here:

cat /proc/scsi/scsi
1
Error "malformed worker response".
  1. Open SSH terminal.
  2. Run the following command:
cd /var/www/keitaro/application
find . -type f -name "*.php" -exec sed -i 's/?>//g' {} +
1
2

# Optimizing database

Open the Maintenance — Status section in Keitaro. If there is a Clickhouse upgrade notification, contact support for the transition to Clickhouse. If the Maintenance — Status looks like this, Clickhouse is installed, and optimization is not required.

# Deleting all the statistics

kctl run cli-php system:delete_mysql_stats
1

The command will ask for confirmation to remove all statistics from MYSQL:

[WARNING] Important! The following will happen:
           - old versions tables removal,
           - clicks stats to the exact date removal (if specified in parameters),
           - optimization of the clicks in the stats,
           - cleaning all the current stats completely (if specified in parameters).

           These inquiries will block the database
           during the implementation and require additional free space on a disk.
1
2
3
4
5
6
7
8

Type in yes and press ENTER to confirm.

Automatic cleaning.

To turn on automatic data cleaning, set Stats store term in the settings.

← Update Keitaro →