bristol@mailman.lug.org.uk
To subscribe or unsubscribe via the World Wide Web, visit
https://mailman.lug.org.uk/mailman/listinfo/bristol
or, via email, send a message with subject or body 'help' to
bristol-request@mailman.lug.org.uk
You can reach the person managing the list at
bristol-owner@mailman.lug.org.uk
When replying, please edit your Subject line so it is more specific
than "Re: Contents of Bristol digest..."
Today's Topics:
1. Last LUG Meeting of 2016 this Saturday! Tommorow! (Sebastian)
2. Postgres optimised h/w (Martin Moore)
3. Re: Postgres optimised h/w (Chris Boot)
4. Re: Postgres optimised h/w (mark chard)
5. Re: Postgres optimised h/w (mark chard)
----------------------------------------------------------------------
Message: 1
Date: Fri, 16 Dec 2016 08:49:20 +0000
From: Sebastian <sebsebseb_mageia@gmx.com>
To: <sebsebseb_mageia@gmx.com>, <bristol@mailman.lug.org.uk>
Subject: [bristol] Last LUG Meeting of 2016 this Saturday! Tommorow!
Message-ID: <1f1b322e-7803-47d0-b084-8d8c6847b5d8@gmx.com>
Content-Type: text/plain; charset=utf-8; format=flowed
Hi
WOW what a great turn out, WOW, WOW, indeed, what a great turn out, for
last months November LUG meeting. I remember emailing here on the list
about how it would be basically good to have a 10 people turn out, and we
had like 11, including ourselves people who turned up Three from the event
the previous month it seems as well, well ok two don't really count I guess
then since were already on the mailing list apparnatly, but still one guy
did come in from the event. That was good, after running the event and the
type of event it was, and general turn out it ended up getting, and of
what kind of people. The big question would be though, can we have a
about 10 people turn out at the LUG meeting tommorow as well or not? 10
people turn out is rather impressive, since a lot of people don't come to
the meetings anymore or rather rarely, that are still on this list. Its not
just, because quite a few people have moved away from the general, big area
of, Bristol, Bath, and sourunding areas. Yes our LUG may be mostly social
chat now for the meetings, rather than helping people with basic to
advanced support issues, but thats since Desktop Linux is generally much
easier to use now for most distros, than it used to be, and its been like
that for many years now.. I been to the LUG meetings since I think it was
April 2012 I think last month was the biggest turn out of people that
have ever had it in that time, whilst I have been there, or if not for
quite a while.
I would like to meet some of the people who used to come to the LUG
meetings a lot this Saturday ideally, but I think to keep the in person
side of this LUG nicely alive, with good turn outs of people for most LUG
meetings etc, its good now to do things at times that will hopefuly bring
new people into them, such as events. Next year 2017 should have three
events it seems, two Linux Presentaiton Day events, and one bigger event
organised by someone else as well it seems, with public speaking even.
Which would be quite impressive right?
Tommorow is the Christmas meeting at the Knight's Templar hence a week
earlier than usual as well, we should be sitting at the table in the left
hand side corner by the plugs at the back of the pub on the lower level as
usuaul, or near there instead if someone else not us is already there. The
first one of us will probably arrive at about 2pm, but for anyone who may
be new that might be coming tommorow I would suggest not turning up untill
2:30pm by the earliest, or may be there on own. As usaul I hope to meet
new people at the LUG meeting, and I probably won't be there untill about
3pm myself, and then laving late about 6pm/7pm. What about other people
who are intending on coming tommorow? We can eat Christmas dinner there
tommorow as well, already had some last month, but will again ;). Also I
been wondering about Samir and what plan he actusally booked for FOSDEM,
and if our newer person from the event, who is hopefuly on this list now,
is intending on coming still or not as well.
Anyone got some USB's to bring along tommorow? Yes Linux Live USB's!
Ideally of more fun distros to so not just common things like Fedora and
OpenSUSE, and Ubuntu 16.10, but yes please someone bring those along to.
Oh and the USB's I am after must have support for UEFI. I guess the USB's
don't have to be Linux though. You see I have bought a interesting mini
PC, that was going to crowd fund earlier this the year, but didn't reallly
have the spare money then. The pre installed Windows 10 didn't last that
long on there at all, more details about that at the LUG meeting
tommorow. I had a USB stick with Ubuntu 16.04 on it, so put that on.
Ubuntu 16.04 mostly works on it, altough with some hardware issues, because
of how the device was only really intended to be made for Windows 10, but
considering that and how much still just works, that's quite good really.
For a few things can probably fix onself though. More details tommorow at
the LUG meeting.
2016 Regards
Sebastian
--
Sent using Dekko from my Ubuntu device
------------------------------
Message: 2
Date: Fri, 16 Dec 2016 11:01:08 +0000
From: Martin Moore <martinm@it-helps.co.uk>
To: Bristol and Bath Linux User Group <bristol@mailman.lug.org.uk>
Subject: [bristol] Postgres optimised h/w
Message-ID: <175B3D16-5849-45A9-9E75-1B8E8635FDCB@it-helps.co.uk>
Content-Type: text/plain; charset="UTF-8"
Morning all.
I have a potential requirement for a Debian/Postgres 9.3 server that will handle a relatively high number of unindexable queries (co-ord based, I'll look at GIS but not sure it'll help) that take some time – currently about 6 secs with one query (although that is with a lot of redundant data which I'm currently removing). There could be 50k queries/day. Writes will be minimum. I'll look at further tuning the query, but as it's a function I can't seem to get any plan details for it ☹
So, 1st question – will the 4Gb per process limit of the 32bit pae kernel limit the caching of the database which will be in the 10's of Gb.
Are SSD's now at a stage where they're suitable/reliable for this use – am I safer using RAID 1 mdadm with 1 ssd and 1 hdd (how will it know use the ssd for reads?)
And I presume as much RAM as will fit in the box!
Any other thoughts welcome.
Martin.
------------------------------
Message: 3
Date: Fri, 16 Dec 2016 11:33:26 +0000
From: Chris Boot <bootc@bootc.net>
To: Martin Moore <martinm@it-helps.co.uk>, Bristol and Bath Linux User
Group <bristol@mailman.lug.org.uk>
Subject: Re: [bristol] Postgres optimised h/w
Message-ID: <f9306909-a530-3c72-7f22-99d5ea01b1e5@bootc.net>
Content-Type: text/plain; charset="utf-8"
On 16/12/16 11:01, Martin Moore via Bristol wrote:
> Morning all.
>
> I have a potential requirement for a Debian/Postgres 9.3 server that will handle a relatively high number of unindexable queries (co-ord based, I'll look at GIS but not sure it'll help) that take some time – currently about 6 secs with one query (although that is with a lot of redundant data which I'm currently removing). There could be 50k queries/day. Writes will be minimum. I'll look at further tuning the query, but as it's a function I can't seem to get any plan details for it ☹
Hi Martin,
I'm no PostgreSQL expert (particularly when it comes to GIS and indexes)
but it sounds like your biggest issue is figuring out how to index your
data properly. I'd definitely tackle that problem first, before
considering any shiny new hardware.
Assuming this is for a business, you would probably gain a tremendous
amount from hiring a PostgreSQL expert for a few hours' consultancy to
help you figure this out.
> So, 1st question – will the 4Gb per process limit of the 32bit pae kernel limit the caching of the database which will be in the 10's of Gb.
Yes, friends don't let friends use 32-bit Linux in 2016.
> Are SSD's now at a stage where they're suitable/reliable for this use – am I safer using RAID 1 mdadm with 1 ssd and 1 hdd (how will it know use the ssd for reads?)
Yes, SSDs are very reliable, even crappy desktop ones are good. That
being said, for a database server, I'd definitely recommend some of the
Intel DC S3520 (if you're doing more reading than writing) or S3710 (for
a more write-heavy workload).
Adding a hard disk will only serve to slow things down. SSDs are more
reliable than hard disks now too. Just don't.
The *only* downside of SSD now is cost.
I would also recommend a hardware RAID controller for a database: MD
RAID is great but far less flexible, and the more processing and IO you
can farm off to dedicated hardware the faster your database server will go.
> And I presume as much RAM as will fit in the box!
Yes, up to a point. If your database is 10GB and you're not doing that
much with temp tables and so on, 16GB might just be enough to squeeze
the whole thing into RAM. I'd probably go for 32GB and see how it goes.
No point going for 256GB+ if it's going to sit around unused.
> Any other thoughts welcome.
For new hardware, think carefully about CPUs. Intel is the only horse in
the race at the moment, so the choice is essentially Xeon E3 or Xeon E5.
E3 means fewer, faster cores while E5 means more slower cores. If you
don't have a very parallel workload and the 64GB RAM ceiling isn't an
issue, the cheaper E3 will be much faster for you. Faster E5s get
expensive very quickly, but you get lots of cores and can add obscene
amounts of RAM.
<plug>
My day job is designing, building and supporting Linux systems for
businesses. Send me a mail off-list if you're interested in what we can
do for you.
</plug>
Hope this helps!
Cheers,
Chris
--
Chris Boot
bootc@bootc.net
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 949 bytes
Desc: OpenPGP digital signature
URL: <https://mailman.lug.org.uk/mailman/private/bristol/attachments/20161216/b73edce1/attachment-0001.sig>
------------------------------
Message: 4
Date: Fri, 16 Dec 2016 11:44:27 +0000
From: mark chard <machard.1984@gmail.com>
To: Bristol and Bath Linux User Group <bristol@mailman.lug.org.uk>,
Martin Moore <martinm@it-helps.co.uk>
Subject: Re: [bristol] Postgres optimised h/w
Message-ID:
<CAA7DJ2k=s+8v6ok0X+8D+BK_j+izYs2KL0U9=yvef4KFaSzR6g@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"
Ssd's are a good move but be careful which you purchase with as read/write
throughput may be a restrictive factor. In my mind a nice RAID6 array of
10k 6GB/s SAS disks might be what you are looking for here.
On 16 Dec 2016 11:03 am, "Martin Moore via Bristol" <
bristol@mailman.lug.org.uk> wrote:
Morning all.
I have a potential requirement for a Debian/Postgres 9.3 server that will
handle a relatively high number of unindexable queries (co-ord based, I'll
look at GIS but not sure it'll help) that take some time – currently about
6 secs with one query (although that is with a lot of redundant data which
I'm currently removing). There could be 50k queries/day. Writes will be
minimum. I'll look at further tuning the query, but as it's a function I
can't seem to get any plan details for it ☹
So, 1st question – will the 4Gb per process limit of the 32bit pae kernel
limit the caching of the database which will be in the 10's of Gb.
Are SSD's now at a stage where they're suitable/reliable for this use – am
I safer using RAID 1 mdadm with 1 ssd and 1 hdd (how will it know use the
ssd for reads?)
And I presume as much RAM as will fit in the box!
Any other thoughts welcome.
Martin.
_______________________________________________
Bristol mailing list
Bristol@mailman.lug.org.uk
https://mailman.lug.org.uk/mailman/listinfo/bristol
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mailman.lug.org.uk/mailman/private/bristol/attachments/20161216/9cf3b724/attachment-0001.html>
------------------------------
Message: 5
Date: Fri, 16 Dec 2016 11:45:59 +0000
From: mark chard <machard.1984@gmail.com>
To: Bristol and Bath Linux User Group <bristol@mailman.lug.org.uk>,
Martin Moore <martinm@it-helps.co.uk>
Subject: Re: [bristol] Postgres optimised h/w
Message-ID:
<CAA7DJ2mFzrLyi240z4O8SzbAqRn9k0kXSvVdBcNYfWwpGvi-uQ@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"
Why are you using a 686-pae kernel?
On 16 Dec 2016 11:03 am, "Martin Moore via Bristol" <
bristol@mailman.lug.org.uk> wrote:
> Morning all.
>
> I have a potential requirement for a Debian/Postgres 9.3 server that will
> handle a relatively high number of unindexable queries (co-ord based, I'll
> look at GIS but not sure it'll help) that take some time – currently about
> 6 secs with one query (although that is with a lot of redundant data which
> I'm currently removing). There could be 50k queries/day. Writes will be
> minimum. I'll look at further tuning the query, but as it's a function I
> can't seem to get any plan details for it ☹
>
> So, 1st question – will the 4Gb per process limit of the 32bit pae kernel
> limit the caching of the database which will be in the 10's of Gb.
>
> Are SSD's now at a stage where they're suitable/reliable for this use – am
> I safer using RAID 1 mdadm with 1 ssd and 1 hdd (how will it know use the
> ssd for reads?)
>
> And I presume as much RAM as will fit in the box!
>
> Any other thoughts welcome.
>
>
> Martin.
>
>
>
>
>
>
>
>
> _______________________________________________
> Bristol mailing list
> Bristol@mailman.lug.org.uk
> https://mailman.lug.org.uk/mailman/listinfo/bristol
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mailman.lug.org.uk/mailman/private/bristol/attachments/20161216/1d04f79d/attachment.html>
------------------------------
Subject: Digest Footer
_______________________________________________
Bristol mailing list
Bristol@mailman.lug.org.uk
https://mailman.lug.org.uk/mailman/listinfo/bristol
------------------------------
End of Bristol Digest, Vol 673, Issue 4
***************************************
Tidak ada komentar:
Posting Komentar