bristol@mailman.lug.org.uk
To subscribe or unsubscribe via the World Wide Web, visit
https://mailman.lug.org.uk/mailman/listinfo/bristol
or, via email, send a message with subject or body 'help' to
bristol-request@mailman.lug.org.uk
You can reach the person managing the list at
bristol-owner@mailman.lug.org.uk
When replying, please edit your Subject line so it is more specific
than "Re: Contents of Bristol digest..."
Today's Topics:
1. Re: Postgres optimised h/w (mark chard)
2. Re: Postgres optimised h/w (Steve King)
3. Re: Postgres optimised h/w (Amias Channer)
4. Re: Postgres optimised h/w (Martin)
5. Re: Postgres optimised h/w (mark chard)
----------------------------------------------------------------------
Message: 1
Date: Fri, 16 Dec 2016 12:20:50 +0000
From: mark chard <machard.1984@gmail.com>
To: Bristol and Bath Linux User Group <bristol@mailman.lug.org.uk>,
Martin Moore <martinm@it-helps.co.uk>
Cc: Chris Boot <bootc@bootc.net>
Subject: Re: [bristol] Postgres optimised h/w
Message-ID:
<CAA7DJ2kHXn6EKfSVvtp5rbsxoMyjS7Y8M+oH-oRtFotfYyaong@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"
It all depends on which card you buy. I like my LSI cards. Research is very
important.
On 16 Dec 2016 12:17 pm, "Martin Moore via Bristol" <
bristol@mailman.lug.org.uk> wrote:
> OK, 64 bit it is!
> Cost isn't a major issue for the drives – plus they don't need to be that
> big so I'll look at those mentioned.
> Are most decent RAID controllers admin-able from a running system or is it
> a BIOS level thing still? Also need how-swap.
>
> According to Wikipedia, RAID6 is no faster at reading the RAID1 – in which
> case RAID1 would be fine. Does this sound right?
>
>
>
> On 16/12/2016, 11:33, "Chris Boot" <bootc@bootc.net> wrote:
>
> Hi Martin,
>
> I'm no PostgreSQL expert (particularly when it comes to GIS and
> indexes)
> but it sounds like your biggest issue is figuring out how to index your
> data properly. I'd definitely tackle that problem first, before
> considering any shiny new hardware.
>
> Assuming this is for a business, you would probably gain a tremendous
> amount from hiring a PostgreSQL expert for a few hours' consultancy to
> help you figure this out.
>
> > So, 1st question – will the 4Gb per process limit of the 32bit pae
> kernel limit the caching of the database which will be in the 10's of Gb.
>
> Yes, friends don't let friends use 32-bit Linux in 2016.
>
> > Are SSD's now at a stage where they're suitable/reliable for this
> use – am I safer using RAID 1 mdadm with 1 ssd and 1 hdd (how will it know
> use the ssd for reads?)
>
> Yes, SSDs are very reliable, even crappy desktop ones are good. That
> being said, for a database server, I'd definitely recommend some of the
> Intel DC S3520 (if you're doing more reading than writing) or S3710
> (for
> a more write-heavy workload).
>
> Adding a hard disk will only serve to slow things down. SSDs are more
> reliable than hard disks now too. Just don't.
>
> The *only* downside of SSD now is cost.
>
> I would also recommend a hardware RAID controller for a database: MD
> RAID is great but far less flexible, and the more processing and IO you
> can farm off to dedicated hardware the faster your database server
> will go.
>
> > And I presume as much RAM as will fit in the box!
>
> Yes, up to a point. If your database is 10GB and you're not doing that
> much with temp tables and so on, 16GB might just be enough to squeeze
> the whole thing into RAM. I'd probably go for 32GB and see how it goes.
> No point going for 256GB+ if it's going to sit around unused.
>
> > Any other thoughts welcome.
>
> For new hardware, think carefully about CPUs. Intel is the only horse
> in
> the race at the moment, so the choice is essentially Xeon E3 or Xeon
> E5.
> E3 means fewer, faster cores while E5 means more slower cores. If you
> don't have a very parallel workload and the 64GB RAM ceiling isn't an
> issue, the cheaper E3 will be much faster for you. Faster E5s get
> expensive very quickly, but you get lots of cores and can add obscene
> amounts of RAM.
>
> <plug>
> My day job is designing, building and supporting Linux systems for
> businesses. Send me a mail off-list if you're interested in what we can
> do for you.
> </plug>
>
> Hope this helps!
>
> Cheers,
> Chris
>
> --
> Chris Boot
> bootc@bootc.net
>
>
>
>
>
> _______________________________________________
> Bristol mailing list
> Bristol@mailman.lug.org.uk
> https://mailman.lug.org.uk/mailman/listinfo/bristol
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mailman.lug.org.uk/mailman/private/bristol/attachments/20161216/7a12a829/attachment-0001.html>
------------------------------
Message: 2
Date: Fri, 16 Dec 2016 16:26:07 -0000
From: "Steve King" <debian@invux.com>
To: "Bristol and Bath Linux User Group" <bristol@mailman.lug.org.uk>
Subject: Re: [bristol] Postgres optimised h/w
Message-ID:
<c37a8cb6a1ab76e51eb029483f7b3715.squirrel@dazzle.invux.com>
Content-Type: text/plain;charset=iso-8859-1
>
> I would also recommend a hardware RAID controller for a database: MD
> RAID is great but far less flexible, and the more processing and IO you
> can farm off to dedicated hardware the faster your database server will
> go.
>
In what way is MDRAID less flexible than hardware raid?
I have never found hardware raid offers more performance or more
flexibility, plus you are at the mercy of the hardware vendor if you hit a
problem.
I can give you several reasons why MDRAID is more flexible than hardware
raid, including its support for asymmetric SSD/NonSSD: --write-mostly
Plus the cost of the things. And in my experience they don't give much of
a performance increase anyway.
--
Steve
------------------------------
Message: 3
Date: Fri, 16 Dec 2016 16:27:15 +0000
From: Amias Channer <me@amias.net>
To: Chris Boot <bootc@bootc.net>, Bristol and Bath Linux User Group
<bristol@mailman.lug.org.uk>
Subject: Re: [bristol] Postgres optimised h/w
Message-ID:
<CAMgU7XW3ym6PaEP=7SKcF3LsgReZMskCznQDSGR75d2Wu+__wg@mail.gmail.com>
Content-Type: text/plain; charset=UTF-8
On 16 December 2016 at 11:33, Chris Boot via Bristol
<bristol@mailman.lug.org.uk> wrote:
> The *only* downside of SSD now is cost.
i'm not sure i really agree with that, SSD is good for reads but
writes can be a problem when they get old.
There are lots of really crap SSD drives around , i used to do testing
for storage startup and from that experience
i could only recommend the samsung evo or intel drives, nothing else
was reliable enough.
Look for drives that can advertise IOPS for performance not MB's
My rule of thumb is not to trust SSD for longer than 6 months of
storage, be prepared to swap them out
and ensure you have backups. For me the killer app with SSD is using
them as frontend storage for enterprise
grade disks , then you get super speed and long term reliability. In
some cases you also get replication off-site.
These days the cool kids are using NVMe storage which goes direct on
to the pci express bus , i have one in m2 form in my laptop
and its amazing, a solid 550MBs , even bigger burst rates and a lot of
concurrency because its not pretending to be a disk.
You will need a linux kernel newer than 3.3 for support.
> Intel is the only horse in the race at the moment
Sunway looks pretty tasty , a little risky for some though.
Cheers
Amias
------------------------------
Message: 4
Date: Fri, 16 Dec 2016 16:36:50 +0000
From: Martin <inkubus@interalpha.co.uk>
To: bristol@mailman.lug.org.uk
Subject: Re: [bristol] Postgres optimised h/w
Message-ID: <1481906210.29516.60.camel@interalpha.co.uk>
Content-Type: text/plain; charset="UTF-8"
On Fri, 2016-12-16 at 16:26 +0000, Steve King via Bristol wrote:
> >
> > I would also recommend a hardware RAID controller for a database: MD
> > RAID is great but far less flexible, and the more processing and IO you
> > can farm off to dedicated hardware the faster your database server will
> > go.
> >
>
> In what way is MDRAID less flexible than hardware raid?
> I have never found hardware raid offers more performance or more
> flexibility, plus you are at the mercy of the hardware vendor if you hit a
> problem.
^^^^^^^^^^^^^ THISTHISTHIS! What he said!
I have recovered RAID systems (personally and advised people in doing
so) that were theoretically broken past the point of repair. This
required reading the kernel source but we got all of the data back. I
have seen systems built on hardware RAID with a fraction of the problems
leading to complete data loss because the vendor wouldn't even bother
answering questions unless you were bringing them millions in revenue.
Unless you REALLY REALLY need the extra performance (and are sure that
you get it) I'd suggest software RAID.
That said, don't most of the big database servers have a "work directly
from the partitions and balance data as you see fit" mode? Because for
them filesystems are just overhead.
Cheers,
- Martin
------------------------------
Message: 5
Date: Fri, 16 Dec 2016 16:51:40 +0000
From: mark chard <machard.1984@gmail.com>
To: Bristol and Bath Linux User Group <bristol@mailman.lug.org.uk>,
debian@invux.com
Subject: Re: [bristol] Postgres optimised h/w
Message-ID:
<CAA7DJ2knb5SZNgXnhBK=cySnVc3xGcyYEPWdJi=_TB4YE0QFPA@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"
A hardware raid removes the overhead from the kernel, has RAM to cache to
on the controller and has a battery for power failure redundancy.
It's all down to which raid card you have and what the underlying system
is.
An i7 may outperform an i386 with a perc5/I.
It all depends on what you have available, what your budget is and what you
would like to do with it.
For example I would be hard pushed to run my 10x3TB RAID6 in kernel.
ZFS does look really promising in anD raid but I have not ventured that far
yet.
On 16 Dec 2016 4:26 pm, "Steve King via Bristol" <bristol@mailman.lug.org.uk>
wrote:
>
> >
> > I would also recommend a hardware RAID controller for a database: MD
> > RAID is great but far less flexible, and the more processing and IO you
> > can farm off to dedicated hardware the faster your database server will
> > go.
> >
>
> In what way is MDRAID less flexible than hardware raid?
> I have never found hardware raid offers more performance or more
> flexibility, plus you are at the mercy of the hardware vendor if you hit a
> problem.
>
> I can give you several reasons why MDRAID is more flexible than hardware
> raid, including its support for asymmetric SSD/NonSSD: --write-mostly
>
> Plus the cost of the things. And in my experience they don't give much of
> a performance increase anyway.
>
> --
> Steve
>
>
> _______________________________________________
> Bristol mailing list
> Bristol@mailman.lug.org.uk
> https://mailman.lug.org.uk/mailman/listinfo/bristol
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mailman.lug.org.uk/mailman/private/bristol/attachments/20161216/757ce5ca/attachment.html>
------------------------------
Subject: Digest Footer
_______________________________________________
Bristol mailing list
Bristol@mailman.lug.org.uk
https://mailman.lug.org.uk/mailman/listinfo/bristol
------------------------------
End of Bristol Digest, Vol 673, Issue 7
***************************************
Tidak ada komentar:
Posting Komentar