How to tune my Postgres server?


How to tune my Postgres server?



I have a 4 GB server with 80GB hard disk. The website is really slow when switching between pages. I checked top processes and postgres processes with 70 -100 % cpu usage keep popping up. they dont last for more than 4 seconds usually and then its a new one. In my postgres database table i have about 12 tables but one of them has about 90 million entries( rows). This is the table causing the slowdown. But how do i tune my postgres parameters like shared_buffer size and all for optimum performance?



It runs on Ubuntu 16 OS, its a django webapp.



my database name is roctim it has size 15 gigabites (as of now, but is increasing)


postgres=# select datname, pg_size_pretty(pg_database_size(oid)) as db_size from pg_database;

datname | db_size
-----------+---------
template1 | 6857 kB
template0 | 6857 kB
postgres | 6992 kB
roctim | 15 GB



the size of my biggest table is 15 GB called "Webapp_sensordata". It collects and stores livedata from machines like crushers, conveyors etc.



When I run select query from pg_stat_statements order by total_time desc;,
I get


select query from pg_stat_statements order by total_time desc;


SELECT "Webapp_sensordata"."id", "Webapp_sensordata"."timestamp",
"Webapp_sensordata"."value", "Webapp_sensordata"."machine_id",
"Webapp_sensordata"."type_id"
FROM "Webapp_sensordata"
WHERE "Webapp_sensordata"."machine_id" = ?
ORDER BY "Webapp_sensordata"."timestamp" DESC
LIMIT ?



and


SELECT "Webapp_sensordata"."id", "Webapp_sensordata"."timestamp",
"Webapp_sensordata"."value", "Webapp_sensordata"."machine_id",
"Webapp_sensordata"."type_id"
FROM "Webapp_sensordata"
WHERE ("Webapp_sensordata"."type_id" = ?
AND "Webapp_sensordata"."machine_id" = ?)
ORDER BY "Webapp_sensordata"."timestamp" DESC
LIMIT ?



as the top two queries, they both access the big table sensor data.



Also my shared_buffer and working_mem is set to the default right now (128mb and 4mb)



When I ran EXPLAIN ANALYZE the second query shown above, i got


QUERY PLAN
-----------------------------------------------------------------------------
-----------------------------------------------------------------------------
----------------------
Sort (cost=62127.86..62135.64 rows=3111 width=24) (actual
time=781.051..781.230 rows=2860 loops=1)
Sort Key: "timestamp" DESC
Sort Method: quicksort Memory: 320kB
-> Index Scan using "Webapp_sensordata_machine_id_e353fc5a" on
"Webapp_sensordata" (cost=0.57..61947.37 rows=3111 width=24) (actual
time=4.190..779.783 rows=2860 loops=1)
Index Cond: (machine_id = 3)
Filter: (type_id = 1)
Rows Removed by Filter: 31440
Planning time: 4.572 ms
Execution time: 781.449 ms
(9 rows)





Welcome to stackoverflow.com. Please take some time to read the help pages, especially the sections named "What topics can I ask about here?" and "What types of questions should I avoid asking?". Also please take the tour and read about how to ask good questions. Lastly please learn how to create a Minimal, Complete, and Verifiable Example.
– Bear Brown
Jun 28 at 13:41





I tried my best to explain with all the details i have. I'm not a database expert, if the answer needs more information im glad to post it as soon as i know what more is needed
– user6714507
Jun 28 at 13:42





Well, to help you people need to know more - which OS you have? Add size of your database - using for example select datname, pg_size_pretty(pg_database_size(oid)) as db_size from pg_database. Add size of your biggest table - using for example select ns.nspname, c.relname, pg_size_pretty(pg_total_relation_size(c.oid)) as table_size from pg_class c join pg_namespace ns on c.relnamespace=ns.oid where relkind='r' order by table_size desc.
– JosMac
Jun 28 at 14:08



select datname, pg_size_pretty(pg_database_size(oid)) as db_size from pg_database


select ns.nspname, c.relname, pg_size_pretty(pg_total_relation_size(c.oid)) as table_size from pg_class c join pg_namespace ns on c.relnamespace=ns.oid where relkind='r' order by table_size desc





You need to find the slow statements, run EXPLAIN (ANALYZE, BUFFERS) on them and add the result to the question.
– Laurenz Albe
Jun 28 at 14:37


EXPLAIN (ANALYZE, BUFFERS)





@JosMac I've added the results from your queries.
– user6714507
Jun 29 at 9:00




1 Answer
1



Your query would be faster if both conditions can be handled during the index scan by a multi-column index:


CREATE INDEX ON "Webapp_sensordata" (type_id, machine_id);






By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Comments

Post a Comment

Popular posts from this blog

paramiko-expect timeout is happening after executing the command

paramiko-expect timeout is happening after executing the command Can someone help me why timeout is happening after executing the command .I am trying to SSH to a machine and execute a command.I want to store the result of the executed command . I have referred paramiko-timeout but i didn't get my expected result. import traceback try: import paramiko except ImportError: raise Exception("Please install paramiko , pip install paramiko") import pexpect from paramiko_expect import SSHClientInteraction def main(): HOSTNAME = "minion5.net" USERNAME = "root" PASSWORD = "help@432" PROMPT = "root@*:~$s+" try: client = paramiko.SSHClient() client.load_system_host_keys() client.set_missing_host_key_policy(paramiko.AutoAddPolicy()) client.connect(hostname=HOSTNAME, username=USERNAME, password=PASSWORD) with SSHClientInteraction(client, timeout=20, display=True) as interact: ...
Read more

Export result set on Dbeaver to CSV

Export result set on Dbeaver to CSV Normally I use Dbeaver for windows and always export my result set like this: run my query --> select the result --> export the result set --> select export to clipboard --> done This step by step puts my result set in my clipboard and I can paste it wherever I want to work with it. The problem is that now I am using dbeaver for mac and this guide is not working. I can go on until the moment that I select my result set like in the image below: exporting data set But once I go further in the process, in the last step I get: no query Note that in "source" it was suppose to show the query that originated the result set, but instead it says just "select. As a result it does't select my result or anything (besides being "successful"). Normally my query would show up there automatically and I couldn't find any option that corrects this problem in the menus. Please help me! Thanks. Aft...
Read more

Opening a url is failing in Swift

Opening a url is failing in Swift breakpointing inside the callback and logging success shows false. I added itms-apps to my plist: LSApplicationQueriesSchemes and put this in my app delegate: func application(_ app: UIApplication, open url: URL, options: [UIApplicationOpenURLOptionsKey : Any] = [:]) -> Bool { return true } func moreTapped() { let url = NSURL(string: "itms-apps://itunes.com/developer/quantum-productions/id979315877") if #available(iOS 10.0, *) { UIApplication.shared.open(url! as URL, options: [:], completionHandler: {(success: Bool) in }) } else { UIApplication.shared.openURL(url! as URL) } } Are you testing on a real device? – rmaddy Jun 21 at 23:57 I try on a device, get "Cannot Connect to App Store". Device is online ...
Read more