# Asus Z9PE-D8 Owner's thread



## TheBlademaster01

Asus Z9PE-D8 WS





Asus Z9PE-D8 WS website
http://www.asus.com/Motherboards/Z9PED8_WS/#overview

Motherboard features & specs


Spoiler: Warning: Spoiler!



CPU
Intel® Socket 2011 Processors
Dual Intel® Socket 2011 for Xeon® processor E5-2600/E5-2600 v2 product family
Supports Intel® Turbo Boost Technology 2.0
* Refer to www.asus.com for CPU support list
Chipset
Intel® C602
Memory
8 x DIMM, Max. 64GB, DDR3 2133(O.C.)/2000(O.C.)/1866/1600/1333/1066 MHz ECC, Non-ECC, Un-buffered Memory *1
Quad Channel Memory Architecture
*For Registered Memory, Max. 256GB DDR3 1866/1600/1333/1066/800 MHz
* Refer to www.asus.com for the Memory QVL (Qualified Vendors Lists).
System Bus
8.0/7.2/6.4 GT/s Intel® QuickPath Interconnect
Multi-GPU Support
Supports NVIDIA® 4-Way SLI™ Technology
Supports AMD Quad-GPU CrossFireX™ Technology
Expansion Slots
4 x PCIe 3.0/2.0 x16 (dual x16 or quad x8) *2
2 x PCIe 3.0/2.0 x16 *2
1 x PCIe 3.0/2.0 x16 (x8 mode) *2
Storage
Intel® C602 chipset :
2 x SATA 6Gb/s port(s), blue
8 x SATA 3Gb/s port(s), black
Support Raid 0, 1, 5, 10
Marvell® PCIe 9230 controller :
4 x SATA 6Gb/s port(s), gray
Support Raid 0, 1, 10
LAN
Intel® 82574L, 2 x Gigabit LAN Controller(s)
Audio
Realtek® ALC898 8-Channel High Definition Audio CODEC
- Supports : Jack-detection, Multi-streaming, Front Panel Jack-retasking
Audio Feature :
- Absolute Pitch 192kHz/ 24-bit True BD Lossless Sound
- DTS Ultra PC II
- Optical S/PDIF out port(s) at back panel
USB Ports
ASMedia® USB 3.0 controller :
4 x USB port(s) (2 at back panel, , 2 at mid-board)
Workstation Feature
Quick Gate: 2 x vertical USB 2.0 on board
ASWM Enterprise
ASMB6-iKVM Remote Management Tool
Special Features
ASUS Digital Power Design :
- Industry leading Digital 7 +1 Phase Power Design
- Industry leading Digital 2 Phase DRAM Power Design
ASUS Exclusive Features :
- Front Panel USB 3.0 Support
- ASUS SSD Caching II
ASUS Quiet Thermal Solution :
- Stylish Fanless Design Heat-pipe solution
ASUS EZ DIY :
- ASUS CrashFree BIOS 3
- ASUS EZ Flash 2
- ASUS MyLogo 2
- Multi-language BIOS
ASUS Q-Design :
- ASUS Q-Shield
- ASUS Q-Code
- ASUS Q-Slot
- ASUS Q-DIMM
Back I/O Ports
1 x PS/2 keyboard/mouse combo port(s)
2 x LAN (RJ45) port(s)
2 x USB 3.0
6 x USB 2.0
1 x Optical S/PDIF out
6 x Audio jack(s)
Internal I/O Ports
1 x USB 3.0 connector(s) support(s) additional 0 USB 3.0 port(s)
2 x USB 2.0 connector(s) support(s) additional 4 USB 2.0 port(s)
2 x USB 2.0/1.1 vertical port(s)
2 x COM port(s) connector(s)
2 x IEEE 1394a connector(s)
2 x CPU Fan connector(s) (2 x 4 -pin)
6 x Chassis Fan connector(s)
1 x S/PDIF out header(s)
1 x 24-pin EATX Power connector(s)
2 x 8-pin ATX 12V Power connector(s)
1 x 4-pin EZ_PLUG Power connector(s)
1 x ASMB6-iKVM connector(s)
1 x VGA connector(s)
1 x Front panel audio connector(s) (AAFP)
1 x AUX panel header(s)
1 x SMBus header(s)
1 x System panel(s)
1 x Power-on button(s)
1 x Reset button(s)
1 x Clear CMOS jumper(s)
Accessories
User's manual
ASUS Q-Shield
2 x COM port cable(s)
8 x SATA 3Gb/s cable(s)
6 x SATA 6Gb/s cable(s)
1 x 3-Way SLI bridge(s)
1 x 4-Way SLI bridge(s)
1 x SLI bridge(s)
0 x 1-port 1394a module(s)
BIOS
64 Mb Flash ROM, UEFI BIOS, PnP, DMI2.0, WfM2.0, SM BIOS 2.6, ACPI 2.0a, Multi-language BIOS, ASUS EZ Flash 2, ASUS CrashFree BIOS 3
Manageability
WfM 2.0, DMI 2.0, WOL by PME, WOR by PME, PXE
Support Disc
Drivers
ASUS Utilities
ASUS Update
Operating System
Windows® Server 2008 R2 Enterprise
Windows® Server 2008 Enterprise SP2 64-bit
Windows® 8 64-bit
Windows® Server 2012
Windows® 7 32/64-bit
Form Factor
EEB Form Factor
12 inch x 13 inch ( 30.5 cm x 33 cm )
Note
1* E5-2600 series CPU support DDR3-2133(O.C)/2000(O.C)/1866(O.C)/1600/1333/1066 MHz
1* E5-2600 V2 series CPU support DDR3-1866/1600/1333/1066 MHz
*2: This motherboard is ready to support PCIe 3.0 SPEC. Functions will be available when using PCIe 3.0-compliant devices. Please refer to www.asus.com for updated details.



E5 ES compatibility listing


Spoiler: Warning: Spoiler!



https://docs.google.com/spreadsheet/pub?key=0Aoo2y0sXe9fUdGROYS1xM2pma0hmbW03UHpOQVpPTXc&output=html



Troubleshooting


Spoiler: Warning: Spoiler!



Quote:
Originally Posted by BlackenedTush View Post

Coolers too tight...?
I mentioned to PhxTriode that my coolers are Dynatron R17s. I had never installed a Sandy Brdge CPU cooler before and I got a little enthusiastic with the spring loaded mounting screws - I had torked them a tad too tight. As a result I was having problems getting any video signals out of my EVGA GeForce 670 GPU. On a hunch Chris at EVGA told me to try loosening the CPU cooler screws. I did and Voila!, everything worked. Evidently the CPU seats on many motherboards are susceptible to very slight warpage under these conditions... leading to just enough distortion to cause marginal CPU Pin connection in the seat, and many potential symptoms. Just a thot...

Quote:
Originally Posted by BlackenedTush View Post
My GTX670 is the EVGA SuperClocked 4GB w/ Backplane (NVIDIA driver version 9.18.13.1070), It looks like a reference card. Not sure if brand or model makes much difference with our board since it follows the NVIDIA reference design for the most part, but it is generally touchy with these Keplers. My BIOS is v0503.

Fixing Posting and GPU Issue

After putting my beast together I also had problems getting my board to post. Not seeing any 64GB kits out there I had purchased two 32GB Vengeance RAM kits. I assumed that they would play well together. At the time I was clueless as to the cause of the posting issue. But by swapping RAM sticks in positions1 & 2. I was able to get posting to happen; but then my monitors only came to life maybe 2 out of the next 10 attempts and then not at all. My monitors kept complaining of no HDMI signal. I put the GTX670 in various PCIe slots with no better results. EVGA helped me. The problem there turned out to be the fact I had tightened my Dynatron CPU coolers too tight. Evidently the excess tightness was warping the CPU seats on the board just enough to cause PCIe anomalies. So I loosened them up to the point that they weren't pressuring the CPUs at all; but not so much that there was play between the CPUs and seats. I then tugged very gently on the CPUs to equalize pressures and re-tightened the coolers a small amount. My system posted, booted into Windows, and my monitors worked!
After the scramble to get it to post, my GPU had ended up in slot 3, so I moved it to slot 5 where I had planned to put it for better cooling distribution and because it doesn't switch down to 8x based on the presence of other cards in even numbered slots. Further tests confirmed that everything was still working.
Now at this point I was thinking that my problems were only due to CPU seat warping causing issues with the PCIe and RAM channels. I hadn't realized that I had also been dealing with some intermittent problems in the RAM itself.

My Advice

Not knowing what you've done or not and what you know or don't know... I don't wish to insult anyone's intelligence. Given that this board seems haunted at times or at least hormonal, this is what I would do taking each subsequent step if posting doesn't happen:
Clear RTC RAM. Make sure jumper ends up in correct position
Check and verify that CPU coolers are minimally tight
Reduce PCIe occupation to only the GPU. Try GPU in odd-numbered slot 5 or 7 (both are 16x, driven by CPU2, and not subject to speed switching complications)
Try GPU in odd-numbered slot 1 or 3 (both 16x/8x, driven by CPU1, and subject to speed switching complications)
Reduce RAM to a single stick in slot DIMM_A1 (the leftmost slot for for CPU1)
Swap alternate RAM sticks into DIMM_A1
Swap GPU into the alternate PCIe slot (5 or 7)
Reduce to most minimal config: only one CPU, one stick of RAM, and one PCIe card (GPU).
Single CPU must be in seat CPU1 (upper right), Single RAM stick must be in slot DIMM_A1 , and GPU must be in odd-numbered PCIe slot 1 or 3 (where it's driven by CPU1).
Try swapping in other RAM sticks into DIMM_A1 and moving GPU to the other odd-numbered slot

If you get it to post at most minimal config, then populate with additional RAM (from one kit). Begin by making a 2x config - add DIMM_B1 and test it, Then make a 4x config by adding both DIMM_C1 & DIMM_D1 and test. If that works, try adding the second CPU.

Quote:
Originally Posted by AndyE View Post

1) The Q-Codes are listed in the manual and provide at least a solution to know until when the POST run ok. Good that the system is now starting
A good summary for initial boot issues:
http://www.tomshardware.co.uk/forum/261145-31-perform-steps-posting-post-boot-video-problems

2) If memory is "missing" then the first thing I would do is to reset the CMOS memory. Potentially, in trying to get the system starting, you changed essential settings and impacted this part of POST

3) ME is the Intel Management Engine which is part of Intels vPro offering. Basically it is a small computer on the motherboard supervising the operation of the system, even if the main CPUs are turned off - i.e. you can remotely start the Asus MB via this mechanism

4) With default CMOS settings, the MB shows the tarbet base frequency, which is in your case 3100MHz. If perf mode and similar things are changed, it could be different (haven't personally tried it)

5) This is normaly behavior. Even if the workload of AIDA, HWInfo etc is not big, you are executing software on your system. There is no hard affinity between the executing thread in an application abnd the physical core in the socket - shows up as a kind of random pattern. Benchmark software potentiall wakes up the CPU to get the timing operation right. I would *only* be concerned, if the system with good benchmark software under load would not go to either 3100MHz or better 3400 MHz (depending on your cooler). 3400 MHZ is the max frequency for all cores in Turbo mode. For 1-2 core under load the speed can go up to 3800 MHz. Good benchmark software for such a system is Intel's Linpack, be more critical of the results of the usual known benchmark apps - they are not optimized for NUMA boards like yours.

6)
Memory amount: This is normal
Memory is used for all kind of things like I/O devices are mapped into physical memory space. Some utilities see one POV, some have other approaches. 48 MB difference look like a PCI express card is mapped into memory space.

4 x 8 GB memory is a suboptimal configuration for this system. One of the reasons for a dual socket MB is a the increased bandwidth you might get. In this configuration the 16 cores of the 2xE5-2687W will starve for memory bandwidth to achive their regular performance. Basically, you run the 8 core socket with the memory bandwidth of a 4 core i7-3770. Either switch to 8x4 GB or 8x8GB. Systems like this are NOT designed to achieve low latency /high speed for individual threads, for these kind of apps an OC system with fewer cores is the better solution. If you need THROUGHPUT, then this system will shine and and leave all single socket systems in the dust. As an example: It is possible to read a 100GB file within 5 seconds into main memory (this is actually what I am doing with mine). Try this with an OC i7-3770K or i7-3930K, it wouldn't be possible.

To identify possible errors, I'd start with one populated socket, and go with 1 RAM DIMM and start there. Run the system under load for a few hours (linpack i.e.) and check the stability in its minimal config. Move from there by adding more memory Dimms to one socket. If all goes well, THEN add the second. If problems arise, change back to a single socket config with the "faulty" CPU and re-check stability. If stability is there, your setup might have been screwed for dual socket ops (or one CPU is indeed defect. Clear the CMOS and re-check with 2 sockets.

As said, many classic benchmark utilities are "confused" with NUMA systems or if the performance goes in regions beyond the design of the software. Some of them get actual slower (as data has to be moved via the QPI interconnect and there are WAY too many cores to choose from. Software which does not explicitly control the mapping of threads and memory to the indivual sockets and cores, will never achieve top performance on a 32 core NUMA system. Most benchmark utilities are not written in this way, which regularily might people belief that the single socket system gives more bang for the buck. This is correct if the assessement is done with these kind of software, it is not correct with properly developed software.

This is especially true for memory bandwidth performance checks .Your decline might come from the utility's inability to properly understand your topology. Check with stream benchmark. A system with 1600MHz memory should be in the 80 GB/sec range


----------



## timmeh87

Hey... I just made a system with this board, and I have a little bit of a problem. There is almost no info out there, so I made an account in order to join this thread. You seem to know your ****, perhaps you can even help me. In order to save myself some time, Im just going to paste the email I wrote to Galaxy

I recently purchased a galaxy GTX 670 and tried to put it into a computer with an ASUS Z9PE-D8 motherboard (latest v503 bios), two Xeon E5 2660 engineering sample CPUS, and 32gb of non-ecc ram. The galaxy card does not work. The fan comes on, it is not detected, there is no video signal.

However... The galaxy card DOES work in my friend's computer. It also DOES NOT work in another friend's (rather old) computer, both of which usually run older GTX cards.

Furthermore, one of those older cards (an EVGA GTX 260) DOES work in my computer. Indicating that the problem lies within the Galaxy 670 card, or the combination of it and the motherboard.

Other info:
- Tried every PCIE slot
- Card is not detected at a very low level, motherboard beeps when onboard video is disabled, indicating no video card found
- Absolutely no issues in another computer
- Monitor does not display the "no connection" warning, indicating that there is a connection, but screen always remains black, no flicker at any time.

I think I should just exchange this card... But what to exchange it for? What works for you?


----------



## DrItanium

First off, I want to thank timmeh87 for identifying that this issue exists with non-EVGA based 670's and the Z9PE-D8 motherboard!

Secondly, I figured out what the problem is. You need to enable "VGA Palette Snooping" in the BIOS and the problem should go away. If it doesn't then it is a combination of settings within my BIOS.

EDIT: Also this board rocks and I love having 24 threads in Linux and Windows.


----------



## PhxTriode

I put a TPC 812 on it, fan hits the RAM.

Im looking for coolers that will not interfere with the RAM or PCIE 1 slot. Any ideas ??

System:

E5-2687w (one for now)
TPC 812
32Gb Corsair 1866Mhz RAM (4 x 8Gb) Can only get 1600Mhz out of them
Cosmos II
Lepa 1600w PSU
Vertex 4 512Gb (Os and programs)
4 1Tb Seagate 6gbs (2 Raid 0's)
Quadro 6000 (got it cheap and plan a Tesla for CS6)

Sorry for the potential stupid question but this is the first system I have built in 15yrs.

Thanks


----------



## xttxtt

I liquid cooled all of mine so not sure what to do with air. If I had to guess I'd say the Zalman CNPS12X or any Zalman CNPS9900MAX series. I also think the Phanteks or the Scythe SCKTN-4000 would be tall enough but I've not worked with them.


----------



## PhxTriode

Quote:


> Originally Posted by *xttxtt*
> 
> I liquid cooled all of mine so not sure what to do with air. If I had to guess I'd say the Zalman CNPS12X or any Zalman CNPS9900MAX series. I also think the Phanteks or the Scythe SCKTN-4000 would be tall enough but I've not worked with them.


Thank you for the reply, I just picked up some H80's and should be here tomorrow. Hope I can get them to fit


----------



## ChrisSibbald

I'm in!

And I have an issue I hope someone can help me with.

Here are my system specs:
ASUS Z9PE-D8 WS MoBo AMI bios 503
Dual Xeon 2687W CPU, each liquid cooled with Corsair Hydro H80
Dual EVGA GTX 580
64GB Kingston ECC Registered DDR3 1300 RAM (8 x 8 GB)
Enermax Platimax 1200 Watt PSU
LSI 9620 4i 4 port SAS/SATA RAID controller
4 x 2TB Seagate Constellation ES
Win 7 Ultimate

I run this box hard, crunching BOINC 24/7 full speed.

My issue:

I am getting a Windows WHEA event 19 frequently. This is a corrected machine check error. Details below:
A corrected hardware error has occurred.

Reported by component: Processor Core
Error Source: Corrected Machine Check
Error Type: Bus/Interconnect Error
Processor ID: 46

The details view of this entry contains further information.

I am not overclocking and have most bios settings to auto.

I turned off Speedstep in bios and the issue went away for four days. However, I want to know what is causing the issue and would like to be able to use speedstep.

Any ideas I can test?

Does anyone know the recommended voltage, frequency and timing settings for this CPU and RAM? I would like to try manual bios settings to see if I can get rid of or identify the cause of this issue.

Thanks in advance.

Cheers,
Chris


----------



## Scorpion49

Quote:


> Originally Posted by *ChrisSibbald*
> 
> I'm in!
> *snip*
> Thanks in advance.
> Cheers,
> Chris


Hey Chris, I saw your post in my thread. I have a few suggestions with BIOS pics but photobucket isn't allowing uploads right now so I will add those later. Does your RAM run at its rated settings in auto mode? I had a very hard time getting my RAM to run properly but I'm not using ECC.


----------



## ChrisSibbald

Thanks Scorpion.

It appears that the bios properly identified the RAM and set timing appropriately. I don't *think* the issue I am having is related to RAM...it is a "bus/interconnect" error. But you never know.

I would really appreciate seeing your bios settings.

My personal email is ChrisSibbald_at_Rogers.com (replace _at_ with @).

Cheers,
Chris


----------



## BlackenedTush

PhxTriode
Did you get the H80's to fit? There's certainly more than enough room up top for the rads in that awesome case of yours, and if I remember right the pumps would nestle right in there between your RAM sticks. I was strongly considering getting your case; but settled on the CoolerMaster HAF X because it fits under my desk. I did order the H70s with Gentle Typhoon fans but then I sent them back because I found that I would have had to fabricate a mounting bracket up top... plus I was also a little concerned with how stiff those Corsair hoses are! It made me a touch apprehensive that the gymnastics required to mount them might possibly weaken them for later cracking. I'm sure they're pretty reliable tho. Cool running too. I settled for air-cooling with the Dynatron R17s. They fit nicely with hi-rise RAM and work just fine. CPU 2 is the hottest due to airflow config and it doesn't get over 50C with heavy rendering or 3DMark11 tests.


----------



## BlackenedTush

Hey ChrisSibbald,

*Coolers too tight...?*
I mentioned to PhxTriode that my coolers are Dynatron R17s. I had never installed a Sandy Brdge CPU cooler before and I got a little enthusiastic with the spring loaded mounting screws - I had torked them a tad too tight. As a result I was having problems getting any video signals out of my EVGA GeForce 670 GPU. On a hunch Chris at EVGA told me to try loosening the CPU cooler screws. I did and Voila!, everything worked. Evidently the CPU seats on many motherboards are susceptible to very slight warpage under these conditions... leading to just enough distortion to cause marginal CPU Pin connection in the seat, and many potential symptoms. Just a thot...

*Mixed RAM kits?*
Sorry, not sure what RAM timings you have. But I do have some experience to share that may or may not generally help. Like you, I opted for 64GB, but probably unlike you, I was naive. When my search didn't turn up any single 64GB kit, I simply purchased two 32GB kits. My assumption was pretty basic... that since all of the RAM in the two kits had been tested at 9-9-9-24, 1T, 1.5v @1600MHZ, then all of the RAM in the two kits will work at that speed. But according to Corsair tech support, my assumption excluded the probability that sticks not tested and validated TOGETHER at a particular speed very likely won't work well together in a computer at that speed. I guess there can easily be incompatibility between kit members that's somehow due to two different integrated memory controllers in the CPUs plus mobo memory controllers next to the DIMM slots - some confusion can happen when they are overwhelmed with the shear number of modules being used (8x8).

As it happens, my system has suffered from instability, corrupted OS, and corrupted apps from the very first successful post. And early on, BIOS indicated that one stick that wasn't registering at all. So I swapped it with an adjacent one and because BIOS now picked it up, I thought things were fixed. But other vague problems, frezing, etc. persisted. Eventually after a month of use that stick acted up again by only registering half of it's capacity. Again swapping helped the BIOS see it, but Speccy and some other apps didn't... go figure.

As I'm sure you know the Auto mode in BIOS picks up several (but not all) of the XMP ratings from the sticks, and it in turn sets the motherboard to work with the RAM at those settings. But only _latenc_y, _timing_, and _voltage_ are honored in that way. _Frequency_ is not. As in my case, BIOS often arbitrarily defaults it to a lower value than rated (e.g., my 1600 RAM was set to work at 1333MHz). But you also know we are provided a way to force the frequency to a handful of common settings. I naively thought that many of my instability issues were due to my RAM not being set exactly at rated speed. Haha. So I forced mine up to 1600. But my problems with instability and corruption only increased (What the heck?!...). I had very confusing symptoms. For instance, occasionally my video driver would fail for a few seconds and then recover. Evidently my assumptions about kits were wrong. Speed rating is a little more particular. And a healthy base of RAM is necessary for a healthy system.

Now not many restarts after forcing my frequency to 1600, it would not boot into Windows, and then after a few more force quits, it wouldn't even post. I went back in and yanked out half of the RAM, making sure to include those sticks that I had swapped earlier. Then it posted and Windows loaded! Testing with Memtest86+ then helped me validate the health of each stick one at a time. That testing showed one big culprit to be the stick I had previously did the swapping with. But I also found one more bad one in that kit, plus a more mildly deficient stick in the other kit (that had remained in the computer).

So evidently my mixing of kits amped up an already bad RAM problem. Corsair tech support told me that if my problems had been simply due to mixing kits, that I could fix it. But I would most likely need to leave my notions that all of the RAM would just get along together at the rated speed. Instead I would need to manually specify some lower settings that all could work together at. They suggested for starters that I go back to 1333 and also move the supply voltage upward a little to stabilize and open things up.

So these days the stakes are higher. DD3 RAM ratings are strictly tied to the kit idea, where groupings of sticks are validated together at a particular speed. Now if a kit works well together at the tested speed, one might get lucky and have a group that could also work well together at a higher clocking. But the more RAM you have, the more "Crucial" the kit philosophy is (no pun intended... )


----------



## FutureX

@timmeh87: I'm having the exact same problem but with an eVGA 670 (FTW 4GB). I tried the "Enable VGA Palette Snooping" but didn't work for me. I've tried what you tried and got the same results, the card works in another machine and the onboard VGA works fine. The monitor kicks on like it's going to work but then I get the "no signal" on the monitor. I've flashed both mobo and 670 BIOS to no avail.

@DrItanium: any suggestions as to what else to try in the BIOS?

I'm running 2 2687w's with 64GB RAM and twin H80's, Corsair ATX1200 PSU.

Thanks!
FutureX


----------



## BlackenedTush

FutureX,

Check out my reply to ChrisSibbald (just above your post). I had the exact same symptoms and it was due to excessive tightening of the CPU cooler. A lot of motherboards are flimsy enough that strong tightening can warp the CPU mounts just enough that the (center?) pins don't contact the CPU affirmatively ... and evidently it's a possibility on this motherboard...

Looser and very even tightening may work for you.
It's always good to have something else to try.

BlackenedTush


----------



## FutureX

BlackenedTush,

Thanks for the tip, it actually worked one time! Then never again...







I loosened both H80s and fired it up and it worked! Then when I restarted the computer it was back to nothing again. so I know this card works in this machine just a lot of weirdness. I also tried other slots and the "VGA palette" to enabled like I've heard. bummer. I even tried loosening the H80s some more but nothing.

FutureX


----------



## anubis1127

Hi Guys, I just ordered one of these boards, and a couple Xeons to go with it. Now I need some help picking out RAM, I don't need a ton of capacity 32gb across the 8 dimms is fine, but I also don't want to break the bank, preferably under $250 for 8 dimms, so I'll be looking at unregistered RAM. Does anybody have any recommendations? The stuff on the QVL is either dated (discontinued), not readily available, or really expensive ($700).

I'll be using the build mainly for folding, so speed is more important than tighter timings. I wanted to go with the Samsung 30nm 2x4gb kits, because I already have one, but I heard that RAM doesn't play nicely with this board, or low-voltage RAM in general, which is unfortunate, because I was going for a low power 2P folding rig.

Any suggestions will be considered, and appreciated.


----------



## BlackenedTush

FutureX,

Come to think about it, when I first had the issue of no signal and hadn't yet figured out the seat warpage issue, I was able to get a signal a few times by switching it among PCIe slots. As recommended for VGA cards, I had originally only focused on blue slot 3, where I was able to get it working the first 2 out of 12 times. Later when I found out about the warpage I also remembered that the CPU control is split between the two CPUs. I never played around enough with the GPU slot location to test seating on CPU2, so it is possible that for me CPU2 was seated okay while CPU1 wasn't.. I'll never know.

That said, there are actually two more related things to try:

1) Even tho you have loosened the screws, there may be a difference between the seat warpage of the respective CPUs that could be exploited - where one CPU may now be seated properly while the other one isn't quite. If that is the case, switching from slot group 1-4 to slot group 5-7 or (vice versa) would tell you that. Note also that CPU1's blue slots (1 & 3) are built to autoswitch from 16x to 8x whenever its black slots (2 & 4) are occupied. I say that because it is possible in the real world, that these two slots react in a more sensitive to marginal connections in the CPU seats.

CPU1 controls slots 1 & 3 (blue, w/ autoswitch), plus slots 2 & 4 (black)
CPU2 controls slots 5 & 7 (blue), plus slot 6 (black)

2) Again even though you loosened the cooler screws, the CPUs may still need re-seating. There are three degrees of effort that you can try to accomplish that end. Thankfully I was successful simply doing "a".:

a. loosen the cooler screws as far as possible without disengaging them from the bracket. Then gently pull the cooler pump away while wiggling it, Then re-tighten marginally.
b. Remove the coolers, reseat the CPUs, replace the coolers, and re-tighten marginally
c. same as "b"; but switch the CPUs

Good luck!

BlackenedTush


----------



## BlackenedTush

I am using 64GB of Corsair Vengeance DDR3 1600. It only uses 1.5 volts. This RAM is on the ASUS Qualified Vendor List (QVL), although it has only been officially tested/qualified at 1300MHz). This list is available from the Z9PE -D8 WS product page on the ASUS website Google "asus z9pe-d8 ws". Click on the middle-of-the-page gray tab called "Memory/Devce Support"., then download the three documents found there.

I originally purchased two 32GB kits (CMZ32GX3M4X1600C10) and filled up all 8 slots. Mixing kits like this is definitely not advisable these days with the speeds that they are pushing RAM to. If you need 32GB, then purchase a single 32GB kit. Ultimately I was very fortunate because 1) I got faulty RAM from Corsair (not common), and 2) my bad RAM sticks were found in BOTH of the kits! That meant that Corsair was obligated to replace both kits entirely. And awesomely enough, they were willing to replace them with one 64GB kit! That 64GB kit works flawlessly BTW.... So I'd recommend Corsair and their Vengeance RAM.

Now lots of RAM sticks these days come with cooling fins that could easily interfere with CPU coolers. My Vengeance RAM is that way. But Corsair also makes a low profile versions. Make sure that your RAM is low profile, or that your cooler is high-rise, or the cooler doesn't extend out over the RAM slots.


----------



## BlackenedTush

annubis 1127,

I forgot to say that the Corsair Vengeance 32GB kit I mentioned above can be had for around $150-$160 on Amazon or Newegg. And while they may have other configurations not on their website, the 32GB Kit that Corsair promotes there doesn't fill all 8 slots... it's 4 x 8GB.

Also you could conceivably mix the RAM you have with some newer RAM; but it will come at a cost. You'd have to play with it in the BIOS to reach the highest common denominator (by upping the voltage and lowering the speed to maybe 1333 or even lower).

And sorry. I see now that you already knew about the ASUS QVL...

BlackenedTush


----------



## anubis1127

Quote:


> Originally Posted by *BlackenedTush*
> 
> annubis 1127,
> I forgot to say that the Corsair Vengeance 32GB kit I mentioned above can be had for around $150-$160 on Amazon or Newegg. And while they may have other configurations not on their website, the 32GB Kit that Corsair promotes there doesn't fill all 8 slots... it's 4 x 8GB.
> Also you could conceivably mix the RAM you have with some newer RAM; but it will come at a cost. You'd have to play with it in the BIOS to reach the highest common denominator (by upping the voltage and lowering the speed to maybe 1333 or even lower).
> And sorry. I see now that you already knew about the ASUS QVL...
> BlackenedTush


Hmm, ok, thanks, that's food for thought. I was hoping to get at least 1866, preferably 2133 out of the RAM, but that may be asking a bit much for the limited budget I have.

Are you using the RAM at its rated speed?

This is the kit I was considering before reading your reply: http://www.newegg.com/Product/Product.aspx?Item=N82E16820231499 2x of those, it's on the QVL, is 1.5v, and resonably priced. Unfortunately they don't have any 8x4gb kits that I can see at least.

That kit got me tempted to try this one: http://www.newegg.com/Product/Product.aspx?Item=N82E16820231499 Which are similar timings / volts to another g.skill 8x8gb kit on the approved list, but I'm not sure.

[edit] Just found the 8x4gb kits, looks tempting: http://www.newegg.com/Product/Product.aspx?Item=N82E16820231502

and the kit on the QVL, but in 8x4gb variety: http://www.newegg.com/Product/Product.aspx?Item=N82E16820231500 OOS on that one, same price as the 2133.

I dislike those heatsinks so badly though.


----------



## FutureX

BlackenedTush,

Thanks for the info! I was talking with ASUS this morning and they agreed about the warpage but also thought that once it's warped it may not go back. I've removed both processors and am letting them sit for a few hours then I'll replace them and swap slots. When I first loosened the H80s the video worked right away but then never again. I've been able to get it to work in Slot 3 but I get this power cycling on boot up. It'll power on for about a second then power off, then power back on for about a second then back off...etc. Sometimes only 1 or 2 times before it continues to boot and sometimes 5-6 times. ASUS says that's hardware related so this could all be the same problem. ASUS also said that slot 3 is controlled by PLX and not a CPU. Hopefully I'm getting closer to a solution!

FutureX


----------



## BlackenedTush

FutureX,

I don't know much about PLX other than it being related to intelligent switching. I assume they are talking about the two "Switch ASM 1480" chips sitting between CPU1 and the first 4 slots. I'm seeing that in the "Z9PE-D8 WS block diagram" found in the manual... but it's still connected to CPU1, and logically a CPU seating issue would still apply. Unfortunately the manual leaves much to be desired in clarity and there's not much help for troubleshooting.

It sounds like you had a more fruitful discussion with ASUS than I ever have tho... Awesome! Maybe it's because you offered up the warpage possibility first. Each call for me was mostly met with an offer to RMA for a replacement, and when I told them the Q-code that it stopped on (not listed in the manual's list), I was simply told that it's a hardware problem. Sounds familiar.

It is a very complex motherboard and PCIe seems to be very sensitive. I even had difficulties just getting it to post at first. I haven't been able to get their PCIe wireless card to work... (either my original purchase or a replacement).

Sometimes you have to strip it down to minimum config with 1 stick of RAM in DIMM_A1 or 2 sticks (A1 & B1), and a single PCIe card (GPU), and clear the CMOS RTC RAM, to rule out RAM anomalies or weirdness in other cards. Then try the GPU in different slots.

It's really about experimentation...
and RMA is a long process if you choose to go there.

You'll get it eventually.

BlackenedTush


----------



## BlackenedTush

anubis1127,

Seems that I've heard of peeps running RAM kits on this mobo to those speeds... I'm sure I've heard of 1866... don't remember if they had to up the v to 1.6...

I am running at the rated speed: 9-9-9-24, 1T @ 1.5v. 1600MHz.

A few things of note:

The BIOS defaults to auto-detect mode, which uses XMP to query your RAM. The timings will detect correctly; but the speed won't. It will start you out at 1333MHz (pretty standard occurrence). So I had to manually bump that up to 1600. It gives you the option of tweaking all of these settings manually if you wish.
Whatever you get, if you want to push it, just be prepared to do a lot of trial and error. This mobo is complicated and thus fairly sensitive (two different integrated memory controllers in the CPUs plus mobo memory controllers next to the DIMM slots, which is only complicated by the number of RAM sticks you employ).... just sayin'.
I saw several different ratings for my RAM in the company's advertised specs, and I'm not sure if they were XMP or what. That was confusing. You may see this while shopping.
Your CPUs and GPU(s) will suck a lot more electricity than your RAM will.
As for the heatsinks on RAM... I think that are a tad overrated. A little more about marketing than necessity.
It's off to school. I'll look at your links when I can.

BlackenedTush


----------



## anubis1127

BlackenedTush,

Thanks for the information, good to know regarding the XMP profile, and speed.

I think I will just go with that g.skill 8x4gb kit. This is a "budget" build, so I don't want to spend more on my RAM, than I did on my CPUs.

As for video, I'll probably start with just the on-board VGA, so not too many W going there.

Yeah, I totally agree with you on heatsinks, on my other g.skill kit in my gaming rig I took the heatsinks off, as the plain black pcb looks way better than that gaudy "ripjaws" heatsink they use.

[edit]

Side note, still can't decide if I want a case, or a tech tray / test bench. I think the open air concept would be better for the board, and CPUs, but I like the idea of having it contained in a case, and protected from pets.

For a case I'm considering a White Switch 810, I saw that Scorpion49 had used one on his build, and I thought that looked nice. (Plus it would match my white PSU.)

For a tech tray / test bench I'm considering one of Spotswood's custom tech trays: http://richchomiczewski.wordpress.com/home/trays/

They would both probably cost about the same in the end, I just have to decide which direction I'd like to go in. The tech tray may come out to slightly more by time I add in the options I would want.


----------



## BlackenedTush

annubis 1127,

I don't see the G.Skill 8x4gb kit on the ASUS QVL; but I _do_ see the _64GB kits_ with similar attributes qualified at 2133MHz , which is actually touchier to accomplish. Wow! So you're okay there... and looking at your links, I see a lot of good things said about it's OCing ability. And also, I see evidence of the G.Skill rep trollin to make sure customers are satisfied. Sounds like a no brainer.

I would have liked a tech tray myself but I live in AZ with a large inside pooch (a recipe for dust, hair, and dander). I pretty much need to vacuum my office every day to keep my bottom feeder case from sucking up the farm, and I dust top surfaces of furniture with a vacuum brush about every other day. You may be in a more humid, less dusty environment. The open tech tray wasn't the best choice for that but I would have loved the open access. Looks like the Rich Chomiczewski has an eATX version, which matches the EEB form factor in size; but not mounting screw pattern. Actually, it will match for all but 2 holes. You could drill and tap your own, ask them to, or grind off the male screw portion of the 2 brass standoffs so they attach to the mobo but not the mobo tray.

I think the open config would run cooler overall (if you keep dust from settling on it). I say that because even tho I have the big side fan on my HAFX case, when I remove that side door, the mobo runs about 5 degrees (F) cooler. But my RAID array runs a few degrees hotter.

I think the Switch 810 supports the EEB form factor. That one was on my potentials list.
I hear ya on the graphics wattage suckage....

One question... were you just winking about making sure the RAM doesn't cost more than your CPUs?

BlackenedTush


----------



## anubis1127

Yeah, the 8x4gb kit isn't actually on there, but the 4x4gb kit is, so I figured rather than getting two 4x4gb kits it'd be better to get one 8x4gb kit.

I think I will probably go with the tech tray, supporting artisans is better than buying from a corporation, IMO, and I like the open air concept. I'll probably just add the "top" option he offers to keep the wife's cats away. I will probably have him drill the holes for the EEB form factor.

And LOL, yeah, kind of, I only spent $342 on my chips, so I don't want RAM more expensive than that.


----------



## Spotswood

Quote:


> Originally Posted by *BlackenedTush*
> 
> .... Looks like the Rich Chomiczewski has an eATX version, which matches the EEB form factor in size; but not mounting screw pattern. Actually, it will match for all but 2 holes. You could drill and tap your own, ask them to, or grind off the male screw portion of the 2 brass standoffs so they attach to the mobo but not the mobo tray. ...
> BlackenedTush


I've already accounted for motherboards with "odd-ball" standoff locations by using threaded *spacers*, which when only bolted to the motherboard, still supports it.









-Rich


----------



## anubis1127

Arg, well I got my board installed, and all hooked up last weekend, but it won't boot consistently, and when it does boot it locks up withing two minutes.


----------



## BlackenedTush

Yeah, good solution Spotswood! It does work perfectly fine.


----------



## BlackenedTush

I feel 4 you anubis1127,
I'm assuming that when you say that it won't _boot_ consistently that you are being fairly strict with your terminology, in other words, you are referring to _loading Windows_ rather than _posting_.
If you mean inconsistent _posting_ and you haven't yet read this entire thread; you may want to do that. This board is touchy. I had both problems.


----------



## anubis1127

Quote:


> Originally Posted by *BlackenedTush*
> 
> I feel 4 you anubis1127,
> I'm assuming that when you say that it won't _boot_ consistently that you are being fairly strict with your terminology, in other words, you are referring to _loading Windows_ rather than _posting_.
> If you mean inconsistent _posting_ and you haven't yet read this entire thread; you may want to do that. This board is touchy. I had both problems.


I mean consistently posting, and loading the OS, haha. When it does post it will lock up before loading the OS, sometimes even in the bios. Pretty much if I can get it to post, it will lock up within two minutes.


----------



## BlackenedTush

anubis1127,
I had those symptoms, plus issues with my PCIe slots. In the "first week of life" I found that swapping RAM sticks in their slots did provide some relief of the posting inconsistencies; but I also had symptoms of my monitor not getting a DVI signal from my GPU. Turns out my CPU coolers were a bit too tight, causing unstable seating of CPU(s) in their seats. Symptoms of that were intermittent problems with posting and my graphics card output. Once that CPU cooler issue was solved (second week of life), I began to be able to post consistently and boot fairly consistently. I wanted so bad to believe that my troubles were over...
If I was to do it all over again, I would definitely employ MemTest to check the RAM. I'd do it by building a boot-able MemTest USB drive, then boot the system into it and test the RAM with only 1 stick installed at a time. That approach would have prevented a lot of headaches for me... If you want to read a summary of those headaches, so you have something to laugh about, read further... or not...

In the "second week of life" I decided to try to finish installing little things like my new ASUS wireless PCIe card. Unfortunately that caused Windows to freeze or BSOD, no matter which slot I chose. ASUS RMA'd it but the replacement had the same results.

In week 3 I started to do a lot of benchmarking/monitoring to get a feel for what kind of machine I had created. I employed free apps like PCMark, 3DMark, Prime95, NovaBench, etc. My system seemed to be just as beastly on 3D and rendering as I was expecting it to be. I did some burn-ins with some of the apps; but nothing formal or extensive. But I was beginning to discover errors and corruption in some of my 3dsMax and After Effects files. And Photoshop got corrupted (not normal). And Speccy showed some wierdness with the very same RAM sticks that I had swapped weeks earlier - showing only half of the RAM in those slots while the BIOS was detecting all of it! That was weird and disturbing. So now knowing that the CPU seating wasn't an issue, I tried re-swapping RAM to the original setup. That fixed the Speccy detection issue and luckily there were no boot issues. But it was clear something was up with the RAM and that usage was possibly magnifying those issues. As I did more research I discovered another recommended app called "SiSoftware Sandra". I installed that; but it couldn't even get through a memory test without freezing up. Hmmm...

In week 4, I began to experience more and more problems with booting. Occasionally it would freeze during the initial Windows splash screen. And when that happened it damaged Windows. A few times I was prompted to have it try and repair itself, and a few times that was actually successful. But I began to need to rely on BU Images to recover. It was clear that something was not only wrong but seemingly getting "wronger". It made sense that my BU images were probably at least marginally corrupted by bad RAM. I got an email message that 3DMark11 had been updated and I tried it. It all seemed to go fine; but when it took me to the Futuremark site to post my results, there was an error message indicating that some corruption had prevented my results from being posted. A FutureMark tech told me that the accompanying error code would be indicative of a hard drive or RAM problem.
I was at a point where I began to question all over again which new parts in my installation were actually problem free. But I was also strongly suspecting that bad RAM could logically account for all of the continuing and growing issues.
So in week 5 I began to experiment with MemTest. I installed it and gave it a test drive by initiating a test on all of the installed RAM (64GB) at once. While a shotgun approach such as this is not the recommended approach for any meaningful results, I did quickly begin to see some red-lettered verification that there were some problems. It was obvious that the recommendation leaned more toward building a MemTest boot disk and doing a methodical series of tests on one stick at a time; but I decided that before I went to so much trouble, I would first see what I could see by stressing it out with that full RAM test and doing it overnight. Now the next morning I was a little preoccupied and forgot that I had turned off my monitors for the test. So I mistakenly thought my machine had frozen again. I did a hard shutdown and reboot, Windows was able to repair itself, and while searching the Windows logs for any record of the MemTest results, Windows froze. After that whenever I attempted to restart, it would only give me the BSOD finger. I removed the 32GB RAM kit with the suspect sticks, and PHEW!... it booted right up again. I concluded that the overnight test had stressed the marginal RAM to the point of imminent failure.

Later that week Windows got corrupted again. Rather than going back to my flakey BU image, I sought MS tech support for a repair solution that I had seen advertised. A few hours later it was clear that they could do nothing for me. So I decided to bite the bullet, wipe the drive and reinstall everything. But first, I made sure that I created a boot-able MemTest USB drive to do some meaningful testing. I booted from that and tested all of the RAM by installing/testing only one stick at a time - the way I should have from the beginning! Results showed that the kit I had removed earlier was suffering from 2 extremely bad sticks. And it showed that the 2nd kit had a marginally bad stick. So I reinstalled two good sticks, reinstalled all my software, and contacted Corsair about RAM replacements. Since then all has been stable. Corsair was kind enough to replace both of my 32GB kits and replace them with a single 64GB kit. I was grateful for that!

So 6 weeks from the "start of life", there was no longer any trend towards corruption, posting/booting is stable and dependable, and the consistent beastliness of my dual CPU monster is getting boring... Just how it should be. Maybe it's silly superstition; but I have not tried installing my ASUS wireless card or re-installing Sandy...


----------



## anubis1127

I tried loosening the CPU cooler to the point where its barely tightened, but no change. I'm down to testing with just one CPU, and one dimm slot filled, and still no luck. I've tried unregistered dimms that are on the QVL, and two sticks of Samsung that isn't on the list, and neither made a difference.

I'm starting to wonder if my ES chips aren't compatible with the board, or if there is just an issue with the board. Unfortunately I don't have any other 2011 Xeons to test with the board. The CPUs work fine in my x79 desktop motherboard.


----------



## BlackenedTush

Anubis1127,

You may have something there about the ES chips... there's not much guarantee that they wouldn't have a minor bug or two baked in. And as touchy as the Z9PE-D8 board seems to be, I could easily picture it choking while the less complex x79 works fine with it.

With the steps you took with the RAM and CPU it sounds like either your MoBo has issues or your ES chips are outside of its spec.

It's more than remotely possible that there is a little variation between your ES chips. In that case, you've probably thought about when you're doing the single CPU test, that you're giving each one a turn at it.
ASUS support seemed quite willing (eager even) to RMA my board... if that step interests you at all... It only interested me as a last resort and fortunately I didn't have to take that step.
Now I get your earlier comment about your RAM potentially out-pricing the CPUs...

One other thing... I hate to admit that I did this; but the very first time I tried to post, the Z9 MoBo had insufficient power from the PSU to do so successfully.
It took a little sleuthing before I realized that this board required the normal 24 pin ATX cable plus _TWO_ of the 8-pin auxiliary power cables. I had connected only one of the 8-pins. Not saying you'd do that; but just goin' down the list of easy stuff to check.


----------



## Scorpion49

Quote:


> Originally Posted by *BlackenedTush*
> 
> Anubis1127,
> 
> One other thing... I hate to admit that I did this; but the very first time I tried to post, the Z9 MoBo had insufficient power from the PSU to do so successfully.
> It took a little sleuthing before I realized that this board required the normal 24 pin ATX cable plus _TWO_ of the 8-pin auxiliary power cables. *I had connected only one of the 8-pins*. Not saying you'd do that; but just goin' down the list of easy stuff to check.


Bolded for reference, my board would not even attempt to power on without both of the 8-pin EPS plugs. Also, it did not like the 6-pin PCI-E to 8-pin converter at all, and I ended up buying an NZXT hale90 with 2 EPS plugs on it and it was good to go after that.


----------



## lloyd mcclendon

I just picked one of these up "open box" for $375.







we'll see how "opened" it was first before I purchase anything for it







. I've been eyeing this board up for a very long time and way back in 2010 I started building up for a dual socket, so I'm a bit excited here







I really don't anticipate any problems, I've always had great luck with ASUS boards. This will be dropping into my main rig which I use for a VM host for a highly threaded software development project. I've had this i7 950 pegged while running tests for the past couple days and the RAM maxed, so, time to upgrade.

I'm looking at the E5-2687W as the CPU option, starting with just one for a period of time, see how it goes, and then a second for a kick in the nuts. I would think two of those suckers would laugh at this workload. If only the multiplier was unlocked .. come on intel









I have the manual here to read and based on what I've skimmed through here, I have some careful shopping for RAM to do. I could see this touching 32GB, so I'll probably be in the 48 or 64 range to start. I don't understand, just a couple of years ago, 8GB DIMMs were always super super expensive, and now they are dirt cheap. ?

It had better be fully VT-D compatible this time too, the marvell chipset scares me as the one on my current board is out of spec, but I believe it can be turned off on this one. Is anyone here using the VT-D to passthrough a PCI card to a VM?


----------



## eq2account

Just got a Z9PE-D8 WS today. My issue is similar in terms of no post (not even fan spin or post-code), but there is a difference on mine. I have a new Seasonic 1000w Platinum + Z9 + 2x E5-2690. I've tried 1 million different combinations of things, but have stripped it down to 1 cpu (heatsink + fan) + power supply. You'd think the fans would at least spin or maybe a give me an error code, but nothing except for a really strange light pattern. The on-board power button turns red, then goes off, the onboard reset button turns green then goes off, then a small green led in the bottom corner goes on then off. The cycle repeats over and over. This is when power is simply attached to the board (before I even try to turn it on). This is sitting on a box on my bench with literally nothing else attached. I cant figure it out for the life of me. I'm thinking maybe a fried bios (since if i remove the bios clr jumper the pattern stops). I've taken away power, pulled the cmos bat, jumpered the clr pin then put it all back and the exact same thing. Completely frustrating and making me hate ASUS more and more by the second. Anyone else seen this or have ideas. I plan to RMA with Asus on Monday, but was hoping someone had an idea before it lose another week to shipping.


----------



## BlackenedTush

lloyd,

I didn't anticipate any problems either; but I've had my share







. This thread is full of em. But I'm happy with my board now that things are stable.

Even if unlocked, that E5-2687W is a pretty dang beefy processor (and quite a pricey "kick in the nuts" at that).

I had some issues with the Marvell chipset (can't quite recall right now what exactly it was). I've got 9 disks (correction - 10). I elected to keep my VT-D disabled. And come to think of it, I think I might have left Marvell enabled; but I'm actually avoiding it by employing a PCIe RAID controller for 5 HDDs and then using Intel's SATAs to cover 3 SSDs & 2 HDDs.

Sounds like a good choice to go with a 48GB or 64 GB kit to start with. Can't easily mix and match DIMMS these days, so adding RAM later can be problematic (unless of course you are willing to up the voltage and drop the frequency).

Hey good luck!

BlackenedTush


----------



## BlackenedTush

eq2account,

Sorry about your no post... and thanks for posting... (sorry, I couldn't resist).

I'm having a little difficulty understanding the sequence of what you've tried and what exactly happens as a result; but if the lights are going crazy before ever hitting the power button, it sounds to me like you have a goofy MoBo. But I also wonder if the clr jumper might be the culprit... maybe having come with the clrtc jumper installed in the wrong position to start with, and then after attempting the purge of the CMOS, you have put it back where it started.

Also, you didn't mention RAM when describing your stripped down config. Are we to understand that your stripped config doesn't include it? If so, I can't imagine it posting without it.

And also, you do you have three AIPX power connections from the PSU (two 8-pin and one 24-pin), correct?

BlackenedTush


----------



## lloyd mcclendon

well, I got the board today, this thing is pretty heavy and big. very nice. looks great in person. The manual is tvery hick and there's a boatload more features to understand with it than the boards I am used to.

unfortunatley due to it being open box I am missing the rear IO shield. ..... any ideas for obtaining one? perhaps it was a member here that sent it back and has it somewhere.









I picked up this ram which should work well: G.SKILL Ripjaws Z Series 32GB (4 x 8GB) DDR3 2133 (PC3 17000): http://www.newegg.com/Product/Product.aspx?Item=N82E16820231524

later i'll pick up a second kit along with the second CPU. I have to pick one of them up first ... but I've yet to convince myself that spending that much on a CPU is a good idea.







But no turning back at this point


----------



## eq2account

You couldn't help posting. Good one.

I actually RMA'ed the board back to NewEgg as Asus was taking way way too long for their "Advanced Replacement" process. New Egg was out of stock on Z9 so I just bailed on them altogether. I found a used Z9 (that I had the owner send me pictures of actually running) and my problem was solved. The only thing I am wondering about now is the 0503 bios. There are newer ones, but 0503 is working and I can see that just a slight mistake or reset in bios means clearing bios and starting over. For me that means taking the case apart and I'm not in the mood. Is anyone running the 1107 bios and can anyone think of a good reason to upgrade? I'm not convinced turbo mode is working flawlessly, but i do see that the cpu jumps up to 3.1 (from 3.0) here and there. I have E5-2687w and I was hoping it would hit 3.6 or 3.5 GHz once in awhile. Is there a mb monitor tool like there are for the ASUS RoG boards that will let me check temps and fan speeds and clock?

The original issue with the RMA'ed board was determined to be bios corruption by Asus Tech Support. I got $575 back and picked up the working (but used) board for $300, so it felt like a slight win.


----------



## eq2account

My build:

E5-2687w
4 x 4GB G. Skill TridentX DDR3 2666
AMD R7950 3GB
2 x Samsung 830 256 GB

One additional question I have is regarding memory speeds people are running. If I use force 1600 i get 9-9-9-11 1T, which is very solid and stable. If I use force 1866 (2T) or 2133 (2T) I get constant BSOD. Anyone have any luck at higher memory speeds?


----------



## lloyd mcclendon

e2qaccount, you don't have an extra IO shield laying around from your RMA do you ?







perhaps I am the new owner of your old board through newegg, and I'm missing the IO plate...


----------



## BlackenedTush

eq2account,

Glad to hear about your MoBo deal. Faced with the same dillemma and opportunities, I'd have done what you did.
Your TurboBoost mode experience sounds about right. Those Xeons are locked down to any but the most conservative TurboBoost OCing.
I'm working with the 0503 BIOS myself, and with no problems I've not felt the need to risk it.

Here's the info on the last 4 BIOS releases, taken from on the ASUS downloads page. I see nothing about 1107.
(http://www.asus.com/Motherboards/Intel_Socket_2011/Z9PED8_WS/#download)

*Z9PE-D8 WS Formal BIOS 3206* release (2012.11.07 update)
1. Fixed BIOS System Event Log record with error Date.
2. Adjust Ai Overclock Tuner to support manual mode.
3. Support Manual Mode on each PWM of fan.

* Pls noted if MB BIOS version is older than 3109, users need to flash to version 3109 (rom file) firstly before updating to version 3206 (cap file).

*Z9PE-D8 WS Formal BIOS 3109* release (2012.09.27 update)
1. Support Windows 8 OS.
2. Fixed UEFI OS installation fails.
3. Fix system may hang if NVRAM corrupt.

*Z9PE-D8 WS Formal BIOS 0703* release (2012.08.22 update)
1. Fixed onboard VGA always be disabled when addon VGA card exist.
2. Fixed PCI out of resource issue when all PCI-E slots plug-in all VGA cards.

*Z9PE-D8 WS Formal BIOS 0503* release (2012.05.17 update)
1. Support LSI Mega RAID Function.
2. Add memory ECC enabled/disabled info in BIOS.
3. Update Intel RSTe 3.1.0.1085 Option ROM for SATA & SCU.

BlackenedTush


----------



## eq2account

No extra IO plate sorry. I believe ASUS has an e-store where they sell bios chips and other assorted items. Maybe they have an IO plate for sale.

The RMA'ed board from me is not anything that would be resold (at least I hope newegg.com would not do that to someone).


----------



## ChrisSibbald

Hi eq2account,

I am running a Z9 with two Xeon 2687w and 703 bios and the CPU runs at 3.4 GHz in turbo (see CPU-Z screenshot). Running fine but does still have the occasional WHEA Event 19 Warning that I have never been able to resolve.

Cheers,
Chris


----------



## BlackenedTush

lloyd mcclendon,

That's some good quality RAM...
This thread has supported a decent amount of discussion on RAM, ASUS-blessed RAM lists, and experiences with mixing of kits. By the nature of your comments, it doesn't seem like you've bothered to read them.
Good luck.
BlackenedTush


----------



## lloyd mcclendon

excuse me .. it's on the QVL

perhaps you should read that









I guarantee zero problems. I always seem to be immune to those things ... AKA I know what the F i'm doing.


----------



## miquote93

hi,

Here, I think I have a little trouble with this motherboard. I noticed that my bandwidth PCIe was to 2.5GT / s when normally it is logically 5GT / s.I may be forgetting any adjustment in the bios?
my build:
2 X Intel® Xeon® Processor E5-2665 2.4 GHz
Asus Z9PE-D8 WS (BIOS 3206)
eVGA GTX 570
Seasonic X-Series 760W
Kingston HyperX Genesis 32GB (KHX1600C9D3K8/32GX)
Samsung 830 128 GB

I'd like to have your opinion to solve this problem, apart from this, my setup works very well.

for advance thank you

best regards


----------



## BlackenedTush

lloyd mcclendon,

When you say, "It's on the QVL..." and "Perhaps you should read that", I'm confused.
I never implied that your RAM choice was not on the QVL. In fact I said, "That's some good quality RAM..."

My point was related to your comment about your RAM strategy. You mentioned that you would be starting out with a certain amount of RAM and that at some later date you would be expanding it. My point was that mixing kits like this has been discussed within this thread. It has been discussed as something to avoid... because such a practice on this complicated mobo with today's high speed kits could potentially require those who do so to drop all of the sticks to a lower than advertised speed and to raise the voltage in order to make them compatible with each other.

Such deeper notions are not handled in any of the three QVL lists that cover this motherboard. The QVL, although helpful, is not an exhaustive discourse on best practice regarding all things RAM. In fact nothing like that exists. ASUS has done a limited amount of testing and is depending on users to test this board exhaustively.

I am just saying that it's obvious you haven't read this thread, or you might have brought up kit compatibility, or poo-pooed it, or something... But you didn't even acknowledge the notion. That leads me to believe that either you didn't read it all or you think that such issues only apply to other people.

Again, good luck. I'm not trying to dis you. This thread is about helping each other by discussing our experiences and bouncing ideas off of each other. That's my intent as well.









BlackenedTush


----------



## Liron

The most recent BIOS is great.

I had the problem where the GTX660 was not detected at all in any slot (lots of slots to try!), with or without the VGA palette snoop, with any VGA jumper settings. It also confused me that it was not giving any q-code for it. It went all the way to 62, just like when using the onboard.

Flashed to the 3109 rom and then to the 3206 cap and the card started working flawlessly in slot 3. It probably would work in slot 1 as well, but I haven't tried moving it back up because it's so hard to reach the PCIE latch on that slot with the CPU heatsink so close. I could even turn the VGA palette snoop back off and it continued working fine. The Windows 8 boot disk also stopped going to that super uninformative and unhelpful sad face screen (that was the Q-code 62) and installed without any problem .

Win8 detected everything flawlessly. The only drivers I've even bothered installing so far are the nVidia ones. Even the sound card started playing sound during Windows setup. The Engineering Samples worked great *tears of relief*.

The only problem left were some memory sticks that didn't show up in Windows but they were detected as present in the BIOS. I didn't think reseating would help in that case, but it did.

Everything about the mobo is like a dream so far. And to be able to get 32 threads, 64 GB in quad channel, an 80+ Platinum KW PSU, and 3TB of storage in a nice, slient EATX case for under $2000 would have seemed absolutely insane just a couple of years ago! I could even upgrade the OS to a pro version for just $39.


----------



## Trotador

Hi Liron, could you post what model and stepping are your engineering samples?

I have two 660Ti and many problems to make them work on this motherboard.

Thanks a lot in advance

regards


----------



## Liron

Hi. They are E5 2660 ES 2.3GHz. (Although they actually reach 2.41GHz with speedstep







) Model D Stepping 2.


----------



## BlackenedTush

Good for you Liron!

Thanks 4 the feedback on the BIOS upgrades. I've found the PCIes on this board to be a touch finicky. Nice to know it can be can be tamed a bit.
When I first built my rig I had a new ASUS wireless card and it never worked. It either BSOD'ing or caused Windows 7 to freeze when booting). I RMA'd it for a replacement and I experienced the same symptoms. So I shoved it in a box and built a wired network. And actually I did have some other puzzling anomalies with other cards early on as well.

I'm at v0503 and it's working stably. I've also got an Adaptec RAID controller, which unfortunately isn't UEFI compliant... it boots under the legacy BIOS within v0503 rather than UEFI, so I might have to investigate the potential impact first.

BlackenedTush


----------



## Trotador

Quote:


> Originally Posted by *Liron*
> 
> Hi. They are E5 2660 ES 2.3GHz. (Although they actually reach 2.41GHz with speedstep
> 
> 
> 
> 
> 
> 
> 
> ) Model D Stepping 2.


Thanks, can you precise whether they are B0 (QA8X) or C0(QB7B), I think I had read somewhere that B0's could have problems with .cap BIOS (eg. 3206)

thanks for your help


----------



## Liron

It's not in CPU-Z, but looking at the auction, they are QA90.


----------



## Trotador

Quote:


> Originally Posted by *Liron*
> 
> It's not in CPU-Z, but looking at the auction, they are QA90.


Ok thanks a lot, and you say they are working with BIOS 3206, aren't they? that is great news, mine are QA91 and I was afraid to flash to 3206, well I'm still....


----------



## miquote93

Hi,
I solved my problem with my GTX 570.
for information, i use 2 x E5 2665 ES 2.4Ghz with BIOS 3206. they are C0 (QB7U). everything works perfectly for the moment.

regards


----------



## Liron

Quote:


> Originally Posted by *Trotador*
> 
> Ok thanks a lot, and you say they are working with BIOS 3206, aren't they? that is great news, mine are QA91 and I was afraid to flash to 3206, well I'm still....


Yes, they are. I know how you feel. I was terrified of flashing mine but, since the video card wouldn't work on it, I had no choice.
I hope nobody's mobo gets fried.

Good luck!


----------



## Trotador

Anyone else with B0 ES xeons (QA90, QA91, QA92) and BIOS 3206 working together?


----------



## kweechy

Bought one of these and trying to get up and running with 2 E5 2687W eng samples. Getting Q-Code 62 every time I power on the system. Tried running with only CPU1 active and one stick of RAM, unplugged all HDDs and GPU, tried both Xeons in CPU1 slot...no luck with anything so far.

System won't POST at all, only displays 62 in the LED readout forever while the fans spin.

Later this week, I'll have a chance to drop in some non-ES Xeons and give that a test.


----------



## eq2account

Based on the manual it says this is the "initialization of PCH run time service." What did ASUS Tech Support tell you to try? You might try clearing bios and starting it up again. Ensure you have 2 x EATX 12V connectors (even if you only have 1 CPU)+1 x 24 pin+CPU fans plugged in and memory installed into the correct slot(s). What type of power supply are you running?


----------



## kweechy

Quote:


> Originally Posted by *eq2account*
> 
> Based on the manual it says this is the "initialization of PCH run time service." What did ASUS Tech Support tell you to try? You might try clearing bios and starting it up again. Ensure you have 2 x EATX 12V connectors (even if you only have 1 CPU)+1 x 24 pin+CPU fans plugged in and memory installed into the correct slot(s). What type of power supply are you running?


Going to try to contact them today, been sorting through the problem on my own so far.

Have tried clearing the BIOS though a few times, and always have both of the 8 pin power cables plugged in alongside the 24 pin. I'm running a 1200W Cooler Master PSU right now.

Turns out my Gigabyte UD5 board was running BIOS version F3 and the Xeon 2687W support came in BIOS F10...that wouldn't explain why it's running my 3930K fine since that requires BIOS F7 according to their site.

If I'm feeling ambitious, I'll throw one of those Xeons into the UD5 again and see if I've got any luck this time with it. At least will let me narrow it down to being the Asus board if the Xeons are working.


----------



## kweechy

Update...

No idea what changed really, but my system is now working perfectly. I swapped out the 680 for a barebones Asus card...but was still getting Q-Code 62. Anyway, I plugged in a screen and it was booting, and I can't explain any of it...but I'm a very happy guy suffice to say.


----------



## kweechy

I guess now the real problem...is anyone using this board with a GTX 680? I've got the EVGA 680 FTW 4GB and for the life of me, I cannot get this machine to play nice with it. I get zero video output and I get two different sets of beep warnings when I boot with it. The first is 1 long, 3 short...that is a video card warning. Then I get one long with 8 short later on in the boot process.

The system boots fine with this barebones Asus video card, the type with no PSU connection or fans on it.

Anyway I'm pretty stumped at the moment, and due to my watercooling setup I cannot move the 680 around to try out different PCI slots...though I have tried the barebones card in different slots and it's always fine, doesn't matter where or whether the slot is blue or grey.

This is really annoying because I've just swapped from using a RIVE 3930K as my workstation to using this, and now I'm left without a proper machine to use as my 3930K got downgraded to 32GB RAM and a 560ti in order to give the 64GB and GTX 680 to this Z9PE-D8 machine.

UPDATE:

The card is working fine when the aircooler is on. When I swap it out for the EK block, it stops working again. My only theory is the card's fan header needs a jumper or something, though I haven't tried to run it with the air cooler on, but the fan unplugged.


----------



## BlackenedTush

kweechy,

I'm not sure what you mean by "aircooler"... Is that the CPU cooler?

I had those symptoms myself. I discovered that my CPU coolers were mounted too tight. On this board, this can warp the seats slightly. And heat and cooling can then cause intermittent problems. See earlier posts 11, 16 and 20.

BlakenedTush


----------



## kweechy

Ah sorry, I meant the stock air cooler on the GPU.


----------



## Trotador

kweechy, your 2687w ES are QA91? what BIOS are you using? what PCI slot is your board plugged in?

This board and GTX 660/660Ti/670/680 pairs have repeateadly shown problematic


----------



## kweechy

BIOS is the latest, 3206 I believe or something? Not sure what batch of ES I'm running but that's probably what they are.

My 680 boots more or less fine everytime now that the stock air is back on. Sometimes it's still not recognized, but I'm happy to rule out the board as having the problem, I think it lies with my 680.

Sidenote: how on earth do you change your memory speeds on this board? My 2133 Gskill kit is very sad at 1333 right now.


----------



## Trotador

DRAM speed can be changed in Advanced Menu/Chipset configuration/memory configuration/DDR speed

Do you have CPU-Z installed so you can see the CPU revision (A0,B0.C0,C1,C2?) and the installed BIOS version (3109,3206,3302?) ?

Thanks a lot


----------



## Morachi

Hi guys, new to the forum, although a long time reader so to speak. I've recently put together a rendering HPC and having much the same issue as Kweechy.

The only difference is that i'm running a PNY Quadro K5000 instead of a GTX 680 (however I believe they are both PCIe 3.0 cards using the same Kepler chipset - GK104). Please note, I have made absolutely certain all power connectors (2x 8 EPS etc) are connected. I'm actually typing this from the HPC in question, so the build is solid, just the BIOS config/Firmware is in question. I've found the following (dot pointed for easier reading);

1. The card WILL NOT work in PCIe Slot 1 - period.
2. The card WILL work one out of perhaps 3-4 times in PCIe Slot 3 (but no other slot). This is purely random.
3. When the card fails to be detected I get the same single beep, followed by three beeps at Q-Code 62, then another single beep and 8 short fast beeps.
4. My GTX 480 works flawlessly in ANY PCIe slot.

I have tried the following to correct the issue. Note; nothing has worked so far;

1. Cleared CMOS
2. Updated the BIOS to 3302 (in steps, testing in various PCIe slots each update)
3. Although this is a BIOS POST issue, when booting was successful in PCIe Slot 3, I tried both nVidia Quadro/Tesla drivers 307.45 and 310.70
3. Tried different cards (GTX 480, 690) *Note, the 690 suffers from the PLX issue
5. Tried different slots (Only PCIe Slot 3 works (albiet randomly)
6. Enabled "VGA Palette Snoop", thinking this may force detection (it has worked for some, but not me)
7. Reseated RAM/CPU etc... no luck
8. Contacted PNY for any firmware updates - Their response; sorry this card has no updates - please return the card to the store for a refund (yes really!!!?)
9. Logged a ticket with ASUS - Their response; "Please clear CMOS for a try", "Also, please keep latest firmware of your vga card." (naturally i'd attempted both)
10. Detailed the issue on the ASUS forums - http://vip.asus.com/forum/view.aspx?id=20121229093923207&board_id=1&model=Z9PE-D8+WS&page=1&SLanguage=en-us
11. The PNY Quadro K5000 works flawlessly in any other single CPU board (ASUS P8P67 LE, EVGA X58 Micro)
12. The SMB(?) Error log shows Error 0x55 and the odd 0x00 (I think the latter is from mashing the reset button).

I've also left a review note on the Newegg.com PNY Quadro K5000 product page (as had two others) to warn people that Dual CPU boards don't seem stable enough yet to handle PCIe 3.0 lanes with newer generation GPUs. My guess is that perhaps firmware will be released to resolve this, else none of the newer generation GTX 6XX or Quadro cards will work in the ASUS Dual CPU boards (hell it won't even detect in an EVGA SR-X i'm told). One reviewer says he has the card running flawlessly in PCIe slot 2 (maybe he means x16 slot 2.. which would be PCIe Slot 3? i'd love to know his BIOS config for this!). This is the Newegg.com product review page - http://www.newegg.com/Product/Product.aspx?Item=N82E16814133468&Tpk=quadro%20k5000

The only way I can get the card working is to mash the reset button until the screen throws up the ASUS BIOS splash screen - sometimes I get lucky, others it takes up to 3-4 goes. I will note that once it has booted past the BIOS POST, the platform is rock solid and one hell of a performer! I just don't like to think i've thrown $9000AUD at a HPC that I have to mash the reset button on to get past POST.

System specifications are below;

Mobo - ASUS Z9PE-D8 WS flashed to BIOS 3302
CPU - 2x Intel Xeon ES 2600 series (C1 Stepping) 3.0GHz 8 Core (roughly E5-2687w equivalent)
SSD - 1x Intel 520 480GB (400i Firmware)
GPU - 1x PNY Quadro K5000 4GB
GPGPU - 1x nVidia Tesla K20 (For Maximus 2.0 configuration)
RAM - 64GB (8x8GB) Corsair Vengeance 1600MHz CL9
PSU - XFX Black Edition 1250w (modular, gold rated 80+)
BRD - ASUS BW-12B1ST Blu-ray Burner

If anyone has any ideas as to what options I could try in the BIOS (or anything else for that matter), i'm all ears! (or eyes as it were).


----------



## kweechy

Quote:


> Originally Posted by *Morachi*
> 
> *snip*


Sounds like you and I have identical issues then and damn near identical systems as well...and I believe the Quadro K5000 is just a 4GB 680 like mine but optimized for OpenGL, right?

The REALLY weird thing for me, is that I've had far better luck getting the system to boot when my GPU has the air cooler on it than with the EK block on it. That has been throwing off my variables extremely hard and leading me to believe it's been an issue with the video card's block this whole time.

My system boots 100% flawlessly with an ASUS GT210 video card installed.

Have you recorded your overall success rate with just the K5000 plugged in versus the K5000 + Tesla plugged in?

I still can't wrap my brain around why on earth I've had more success with the air cooler than the blocks...but now I'm trying to remember which PCI lanes I'd been using. My air cooled card is sitting in lane 1, and I'm pretty sure the water cooler attempts were installed to lane 3 only...but then from what you're saying, you had BETTER luck with lane 3, which means my 680 should have been more likely to boot up when I was trying with the EK block attached.

This is all so damn bewildering, made even more so by the fact that once you roll the dice and get the card to boot up, the system is literally 100% flawlessly functioning until it's shut off again and you re-roll.

Quote:


> Originally Posted by *Trotador*
> 
> DRAM speed can be changed in Advanced Menu/Chipset configuration/memory configuration/DDR speed
> Do you have CPU-Z installed so you can see the CPU revision (A0,B0.C0,C1,C2?) and the installed BIOS version (3109,3206,3302?) ?
> Thanks a lot


Hi Trot, I've got the 2687W equivalent Eng Sample Xeons, identical CPU-Z readout as what Morachi has posted. Thanks for the DRAM speed tip by the way, I'd have never in my life found that.


----------



## kweechy

Also, has anyone been able to get any kind of decent clock speed on their memory on this board? I can't get it to boot higher than 1600 so far despite the RAM kit you see in my sig...and when it fails to boot, it fails so hard that it no longer goes into the BIOS, and you painfully have to sit there with the CMOS jumper.

This board can accept some sick hardware, but it sure is a giant disappointment so far considering it's more expensive than most computers out there.


----------



## Trotador

One of Morachi CPUZ shots shows 3312 MHz so it could be a 2890 C0 (QB7S..) with all cores turbo mode ON, the other one shows oddly 3111MHz which could correspond to a 2687w B0 (QA90, QA91, QA92..) with all cores turbo mode ON or a C0 chip with no turbo mode.

Do you know Morachi, Kweechy the Qxxx codes of your cpus?

I have 2687w B0 (QA91), during five months it was working with an old 9600GSO card and BIOS 0303 w/o any issues and DRAM at 2133 MHZ (I had to adjust manually some timing) . Then, I purchased two 660GTi and the problems started. I upgraded to 3109 and entered the nightmare of achieveing that the graphic cards worked in the PCI slots I wanted, maximum DRAM is now 1600 MHz.

To make it short, I feel almost hapy that the PC is working with both cards (PCI1 and 3) and it is being able to do it after rebooting. The PC is dedicated to crunching so it is ON 24/7 and rarely is rebooted. I've had some hangs but it seems pretty stable.

Next, I've got the cable to be able to use the monitor with the on board VGA and will check if I succeed in getting a better combination of operating PCI (7 PCI slots and having to put both cards side by side heatin each other is a pity) or even to add a third card (dream). PCI1 alone is a no go, PCI3 alone works OK, any other combination is also a no go.

I remounted both CPUs, loose the heatsinks etc with no exit either.

I also wanted to flash BIOS 3206 but I read that ES cpus were not working with that version, so the reason of my questions on the codes of your CPUs.


----------



## kweechy

Quote:


> Originally Posted by *Trotador*
> 
> One of Morachi CPUZ shots shows 3312 MHz so it could be a 2890 C0 (QB7S..) with all cores turbo mode ON, the other one shows oddly 3111MHz which could correspond to a 2687w B0 (QA90, QA91, QA92..) with all cores turbo mode ON or a C0 chip with no turbo mode.
> Do you know Morachi, Kweechy the Qxxx codes of your cpus?
> I have 2687w B0 (QA91), during five months it was working with an old 9600GSO card and BIOS 0303 w/o any issues and DRAM at 2133 MHZ (I had to adjust manually some timing) . Then, I purchased two 660GTi and the problems started. I upgraded to 3109 and entered the nightmare of achieveing that the graphic cards worked in the PCI slots I wanted, maximum DRAM is now 1600 MHz.
> To make it short, I feel almost hapy that the PC is working with both cards (PCI1 and 3) and it is being able to do it after rebooting. The PC is dedicated to crunching so it is ON 24/7 and rarely is rebooted. I've had some hangs but it seems pretty stable.
> Next, I've got the cable to be able to use the monitor with the on board VGA and will check if I succeed in getting a better combination of operating PCI (7 PCI slots and having to put both cards side by side heatin each other is a pity) or even to add a third card (dream). PCI1 alone is a no go, PCI3 alone works OK, any other combination is also a no go.
> I remounted both CPUs, loose the heatsinks etc with no exit either.
> I also wanted to flash BIOS 3206 but I read that ES cpus were not working with that version, so the reason of my questions on the codes of your CPUs.


I think what you're seeing in his (and my) CPU-Z shots is that with 1-2 cores active, it goes to 3.3GHz, and with more it'll only cap at 3.1GHz unfortunately. I have no idea if it's possible to get it to be 3.3 all the time, as my board and chips allow me to boot in without tuning anything at 106MHz BCLK...which would give me a speed of 3.5GHz if I could have the chip clock to x33 turbo all the time.

My BIOS version right now is 3206, I'm really surprised that your RAM clocks were affected by a BIOS update though, seems so odd. Wondering if maybe it's worthwhile to try flashing to BIOS 0303 like you'd be using before or something.

Really disappointing all around though...the highest end board that ASUS sells won't play nice with the highest end video cards (after all, what the hell else would you be using if you can drop this cash for the board and CPUs) and won't clock your expensive RAM to the max.

Where can I check out my chip's codes?


----------



## Trotador

Ok on the turbo modes causing the CPUZ screenshots, so if they are all 2687w ES B0 and are working with 3206 or higher is good news even though those BIOS versions seem not to be the solution to the problems we have...

The codes are in the chip case but if they are installed on the motheboard under the cooler you can not see it. If you bought them in the ebay, the code should be in the vendor announce. Otherwise I do not know a software tool that show it, I though CPUZ would do it but according Morachi's screenshots it is not the case since the revision cell is blank. I use linux in my system and can not execute CPUZ.


----------



## kweechy

Quote:


> Originally Posted by *Trotador*
> 
> Ok on the turbo modes causing the CPUZ screenshots, so if they are all 2687w ES B0 and are working with 3206 or higher is good news even though those BIOS versions seem not to be the solution to the problems we have...
> The codes are in the chip case but if they are installed on the motheboard under the cooler you can not see it. If you bought them in the ebay, the code should be in the vendor announce. Otherwise I do not know a software tool that show it, I though CPUZ would do it but according Morachi's screenshots it is not the case since the revision cell is blank. I use linux in my system and can not execute CPUZ.


Any ideas on getting the chips to stay at 33x instead of 31x while turbo'd?

Also, I've run into a bit of a stupid issue on my part...but I setup windows in AHCI harddrive mode and now need to setup my RAID0 array for my storage disks. Is there any way to convert this AHCI main drive so that it will be recognized when I swap the board to RAID mode instead?


----------



## Morachi

Hi guys, the reason for the different speeds is simply because one CPU was in action when the shots were taken, both chips are identical in every other way and only separated by a few FPO numbers







same batch etc (they are both QA91 by the way, B0 stepping - meaning VT-d isn't part of the package, VT-x is though). By the way, both my ES chips worked ok with the BIOS 3302, I think I jumped from the cap file straight to the latest by accident, so I can't confirm if 3209 will bugger the system.

As for testing with the Tesla card, i don't have it as of yet - still on its way to Australia as I type this. As soon as I have it here i'll be giving it a bash to see if I can get it going. As for what PCIe slot it will work in, who knows - given it is also a Kepler PCIe 3.0 based card... i'm horrified to think i've blown $3300 on something that isn't going to play ball. Fingers crossed!

Regarding RAM, I can only force it to 1600Mhz (which is stable I might add), anything else above this isn't - simply a system limitation due to the C602 dual CPU chipset on the mobo as I understand it (CPU code as well). Those who go and buy higher rated ram will only get decent overclocks on single CPU boards. Higher clocked RAM will clock down, it essentially means you've got better bin quality chips on the RAM stick.

I spoke with nVidia thismorning and they agreed that the issue wasn't hardware related as some cards work and others don't in the C602 board, yet all cards work fine in single socket configs. They've escalated it to level 2 tech support who are at present (next few days) going to acquire the board and test it with the card. I made mention that none of the GTX 6XX series or Kepler based Quadro cards (safe saying that as there is only one at the moment) work in the board properly and don't work at all in an SR-X.

PNY were simply put, 100% useless and not customer focused at all. ASUS i've not heard back from since my last communication with them where they suggested updating the firmware and clearing the CMOS. So far my hopes lay with nVidia themselves as this could be an issue going forward for any future K series Quadros of next gen GTX cards.

On the ASUS ROG forums another lad has replied stating his ATI 7950 doesn't work at all in this board either. So it isn't just an nVidia thing. I'm guessing it's sounding more and more like the PCIe 3.0 structure of the card/mobo is the problem here as that is the lowest common denominator.

I have the PR manager from Intel Australia calling me tomorrow regarding the two ES CPU's I have (long story), so if they magically get swapped for retail versions i'll test with those as well, just incase an instruction set is the problem. I'd say at this point it isn't simply due to how many people out there are having the same issue.

The other threads i've posted in are here for visibility (in case I forget to update one on my quest to irritate every vendor i can until this is resolved);

http://vip.asus.com/forum/view.aspx?id=20121229093923207&board_id=1&model=Z9PE-D8+WS&page=1&SLanguage=en-us

http://rog.asus.com/forum/showthread.php?27225-ASUS-Z9PE-D8-WS-Issues-detecting-Quadro-K5000-amp-GTX-6XX-series-GPUs-%28Q-Code-62%29&p=204857&posted=1#post204857

http://www.tomshardware.com/forum/326651-30-asus-geforce#t2170278

http://www.newegg.com/Product/Product.aspx?Item=N82E16814133468&SortField=0&SummaryType=0&PageSize=10&SelectedRating=-1&VideoOnlyMark=False&IsFeedbackTab=true#scrollFullInfo

I've called for anyone with any information to populate the thread in the first link with details of their setup so I can form a solid case to update nVidia with. Hopefully this will help to identify what the problem is and force some responsibility back onto the vendors who at this point have been 100% dismissive, offering no assistance other than "return the item for a refund", with the exception of nVidia.

PS: As for finding the chip codes (FPO numbers) i've included the screen shot of my two CPUs, FPO's circled in red for reference on where to find them. They are etched into the PCB, and I believe only found in the microcode (could be mistaken).



EDIT: Sorry, buggered up, the stepping codes are in the middle of the digit set, not the end







my bad.


----------



## kweechy

Would Intel REALLY swap your chips to be real 2687Ws? How the hell did you manage this?

On a side note, while trying again (in vain I see, after reading your post) to get my RAM running faster than 1600MHz...I did probably 10 full system reboots, and only once did it not recognize my video card.

I'm really itching to try out the card again with the EK block installed, in slot 1 where it is right now and see what happens.

Have you tried jiggling around the card a bit and trying to get it seated a little better in between boot attempts? Might be strictly superstition on my part, but seems like the card boots more regularly now since I've readjusted it.

I've got a post up on the Asus forums as well, but no use so far really. Gotten some posts at least: http://vip.asus.com/forum/view.aspx?id=20121216173040912&SLanguage=en-us&board_id=1&model=Z9PE-D8%20WS&page=2


----------



## Morachi

Quote:


> Originally Posted by *Trotador*
> 
> One of Morachi CPUZ shots shows 3312 MHz so it could be a 2890 C0 (QB7S..) with all cores turbo mode ON, the other one shows oddly 3111MHz which could correspond to a 2687w B0 (QA90, QA91, QA92..) with all cores turbo mode ON or a C0 chip with no turbo mode.
> Do you know Morachi, Kweechy the Qxxx codes of your cpus?
> I have 2687w B0 (QA91), during five months it was working with an old 9600GSO card and BIOS 0303 w/o any issues and DRAM at 2133 MHZ (I had to adjust manually some timing) . Then, I purchased two 660GTi and the problems started. I upgraded to 3109 and entered the nightmare of achieveing that the graphic cards worked in the PCI slots I wanted, maximum DRAM is now 1600 MHz.
> To make it short, I feel almost hapy that the PC is working with both cards (PCI1 and 3) and it is being able to do it after rebooting. The PC is dedicated to crunching so it is ON 24/7 and rarely is rebooted. I've had some hangs but it seems pretty stable.
> Next, I've got the cable to be able to use the monitor with the on board VGA and will check if I succeed in getting a better combination of operating PCI (7 PCI slots and having to put both cards side by side heatin each other is a pity) or even to add a third card (dream). PCI1 alone is a no go, PCI3 alone works OK, any other combination is also a no go.
> I remounted both CPUs, loose the heatsinks etc with no exit either.
> I also wanted to flash BIOS 3206 but I read that ES cpus were not working with that version, so the reason of my questions on the codes of your CPUs.


Hi Trotador, little confused by this comment. You say you've got both 660Ti's working in PCI Slots 1 & 3, however in the following sentence you state PCI1 is a no go? So if i'm reading this right, when using a single card, PCIe slot 1 won't work at all, however with two cards (in PCIe slot 1 & 3) , PCIe slot 1 works ok?

I'd love to know what your BIOS settings were and the Firmware used for this setup, just so I can compare apples for apples when the Tesla K20 arrives?

Also, what was your success rate with dual 660Ti cards in PCIe slots 1 & 3? Did it require multiple resets to detect, or did it work flawlessly on each reboot in that config?

Thanks in advance!


----------



## Morachi

Hi Kweechy, I wriggled, reseated, held the card like a baby amongst other things... seemed fruitless. I too can get really good rolls where it will reset half a dozen times booting flawlessly, then crap itself. I've forced my RAM to 1600MHz as it is 1600 rated, as it defaults to 1333 without forcing even though the XMP profile is detected.

You're right though, the K5000 I believe is just a firmware adjusted GTX 680 for all intents and purposes - and a much higher pricetag! It is also just aircooled, given that the card only chews 122w TDP I guess they didn't see the need for anything grand.


----------



## kweechy

Quote:


> Originally Posted by *Morachi*
> 
> Hi Kweechy, I wriggled, reseated, held the card like a baby amongst other things... seemed fruitless. I too can get really good rolls where it will reset half a dozen times booting flawlessly, then crap itself. I've forced my RAM to 1600MHz as it is 1600 rated, as it defaults to 1333 without forcing even though the XMP profile is detected.
> You're right though, the K5000 I believe is just a firmware adjusted GTX 680 for all intents and purposes - and a much higher pricetag! It is also just aircooled, given that the card only chews 122w TDP I guess they didn't see the need for anything grand.


Yeah it was setting my RAM to 1333 automatically as well, got it set to 1600 now, definitely can't go anything higher than that. What the hell is the point of the SR-X with this chipset and locked Xeons then? Can't OC the chips, can't even OC your RAM...all you get is 4 PCI lanes, which are probably all garbage too!

Quadros generally yeah, they're just different firmware of the consumer counterparts, sometimes with a bit more VRAM added, then charge triple. My studio gave up on Quadros in the last couple years, every machine has a 580 Classified 3GB now. Works great, computes CUDA like a beast, and costs way less than the equivalent Quadro 5000.

Honestly watercooling my build was sheer hobby, not function. These chips don't run terribly hot and the GPUs are built to be happy running at 100% on air. If I leave the radiator fans off though I can get some nice bathwater going on after an hour or so.


----------



## Morachi

Hah! I'll bet, Xeon spa bath, now there's a marketing angle.

The SR-X is an interesting board, it can take up to 96GB of unbuffered DDR3, but that's about the only advantage. It's also buggy as all hell. The only other board by ASUS which has the C602 chipset for consumer workstations is the Z9PE-D16 (on their server site), however I feel it probably also has the same problems.

The E5 Xeons sadly were never built to do the overclocks on, even the ES batches wouldn't handle more than a few ticks past 100 BCLK. The old SR-2 W series Xeons however... different story altogether! Still, the E5's have an average performance increase of 20% over the old generation, so not a bad tradeoff.

The really crappy thing is that the only boards I think that can take the K series in a dual socket config are the OEM providers out there such as Dell and HP. Meaning all the consumer built systems relying on the very few C602 chipsets are going to be sitting about playing with themselves until firmware is released that resolves the issue - be it for the card itself or the motherboards in question. I have my copy of NVFLASH 5.127 at the read (having just fixed the PLX issue with my GTX 690). Given its the same architecture, i'd say the 5.127 should detect the Quadro K series making it a matter of obtaining an updated ROM.

Oh what a kick in the balls this has been. Because both my other half and I use this beast of a machine, we have it plugged solely (well that's the intent) into power and network - VNC'ing to the box. However, if it's not in the same room, or we are remoting to it from another state... well i'm screwed. There aren't any WOL functions i'm aware of that allow for a remote signal to reboot the box, meaning it just sits there beeping like a Peruvian taxi driver.


----------



## Trotador

Quote:


> Originally Posted by *Morachi*
> 
> Hi Trotador, little confused by this comment. You say you've got both 660Ti's working in PCI Slots 1 & 3, however in the following sentence you state PCI1 is a no go? So if i'm reading this right, when using a single card, PCIe slot 1 won't work at all, however with two cards (in PCIe slot 1 & 3) , PCIe slot 1 works ok?
> I'd love to know what your BIOS settings were and the Firmware used for this setup, just so I can compare apples for apples when the Tesla K20 arrives?
> Also, what was your success rate with dual 660Ti cards in PCIe slots 1 & 3? Did it require multiple resets to detect, or did it work flawlessly on each reboot in that config?
> Thanks in advance!


Yes, that's the way it works, PCI 3 works alone, PCI 1 no way. With both cards the system is stable through reboots and does not need any reset to detect both cards.Graphic cards firmware is the one they were shipped with and my BIOS setting are quite standard, I've just disabled some things (serial ports, ..) that I don't use but it was after booting with bot cards as described.

Btw, it is not possible to flash back to an older BIOS version, another oddity of this board...

Quote:


> Originally Posted by *kweechy*
> 
> Any ideas on getting the chips to stay at 33x instead of 31x while turbo'd?
> 
> Also, I've run into a bit of a stupid issue on my part...but I setup windows in AHCI harddrive mode and now need to setup my RAID0 array for my storage disks. Is there any way to convert this AHCI main drive so that it will be recognized when I swap the board to RAID mode instead?


xeon B0 ES max turbo in all cores is x31, no higher multiplier is available

and

http://www.tomshardware.co.uk/forum/269592-14-switch-ahci-format-install


----------



## Morachi

Hi Trotador, can I ask what motherbaord firmware you're currently running? Just purely out of curiousity? I want to log everything so when I get a tech who actually gives a crap I can detail as much as possible for them. On that note, if anyone is having problems, I strongly suggest contacting the tech support/customer loyalty for ASUS/ so it isn't just a single squeaky wheel making the noise.

PS: Had a call from the Australian PR manager this morning talking about replacing one of my CPUs with a genuine one (but they wanted both ES chips back in return) so I said no. Guess I won't be testing with the retail release chips. They've registered the chips to me as now owning them - so 100% legally covered it would appear.


----------



## kweechy

Quote:


> Originally Posted by *Morachi*
> 
> Hi Trotador, can I ask what motherbaord firmware you're currently running? Just purely out of curiousity? I want to log everything so when I get a tech who actually gives a crap I can detail as much as possible for them. On that note, if anyone is having problems, I strongly suggest contacting the tech support/customer loyalty for ASUS/ so it isn't just a single squeaky wheel making the noise.
> PS: Had a call from the Australian PR manager this morning talking about replacing one of my CPUs with a genuine one (but they wanted both ES chips back in return) so I said no. Guess I won't be testing with the retail release chips. They've registered the chips to me as now owning them - so 100% legally covered it would appear.


Any ideas what benefits you get out of legally owning those chips? Bummer about the swap, no such thing as a free lunch I suppose...would cost you the price of 2.5 ES CPUs to get a second 2687W in there to go with the one they send!

@Trotador, thanks for that link, very helpful.

I'll be contacting Asus support tomorrow to make sure our wheel gets squeakier.


----------



## Morachi

Well legally speaking they won't ask for them back! haha. Given I paid less than a 3rd the price of their retail counterparts I can't really complain much.

Given the dual 660Ti setup that Trotador has, i'm very curious as to whether the K5000 + K20 cards in PCIe slots 1 & 3 will work as intended.

On a slightly different tangent, i've found I can get the bclk to run at 103-104 stable, pushing it to 105 will boot fine, but after a bit of intense Cinebench crunching a spontaneous reboot occurs, not sure if that's because the forced 1600MHz RAM scaled with the BCLK to 840Mhz... seems ok at abut 820 though.

Oh, I also tried disabling all the PCIe slots bar no.3 to see if that would force detection - no luck there either. Also re-cleared the CMOS and re-flashed the BIOS with 3302... yep, still at square one. I've also put up a review on newegg.com for the motherboard, which is currently pending approval.

Also, I loaded up GPU Z and found this;



Yes, the card is detected as PCIe 2.0 compliant, however is functioning at PCIe 1.1 - Must research this little gem.

EDIT: Turns out its a feature of the card, running a render test pulls it out of whatever low power mode its in to function at 2.0 speeds - Newegg.com have this card listed as PCIe 3.0!!!? Oh my. False advertising much?


----------



## Morachi

Just an update folks, this is the conversation i've been having with ASUS tech support. I've now been advised that I may have a faulty board and to return it to my sale agent. This is a load of rubbish honestly - that and I had to buy this bloody board from the USA as they aren't selling it ANYWHERE here in Australia anymore... not a big fan of paying stupid amounts in shipping just to be told the board is fine.

As you can see, they have an ASUS GTX 680 running in the top PCIe slot with Firmware 3302 running, however they don't say what the brand of the card is, it isn't one of the cards I own (so useless to me personally), doesn't detail BIOS settings and i'm willing to put money on it that they got lucky first POST and didn't reboot the machine a few dozen times to check for POST boot frequency.

Forwarded message

From:
Date: Wed, Jan 2, 2013 at 8:15 PM
Subject: Re: Re:Re: Motherboard Z9PE-D8 WS [ID=RWTM201212291220234472-671]
To: Cherry Xie

Hi Cherry, I tried to reply however the server says it is busy, so I am replying here.

Can you please detail the following so I can simulate it?

1. Brand of card
2. Any changes to the BIOS setup
3. Type of CPUs used
4. Please tell me how many times you rebooted the machine as it works only SOMETIMES - you may have been very lucky the first time around. As I stated, the board only POSTs 1 in about 4 times.

The board is not faulty as I have tested it with multiple graphics cards;

1. GTX 480 (two individual cards 1x EVGA 1x ASUS) - works in every slot
2. GTX 690 - will only work sometimes (1 in 3-4 times) in PCIe slot 3 (PLX has been flashed to latest firmware)
3. Quadro K5000 - will only work sometimes (1 in 3-4 times) in PCIe slot 3

This is using the BIOS Firmware 3302 with BOTH CPU sockets occupied (Intel E5-2687w CPUs).

Note; all the GPU's listed above work 100% fine in the three other motherboards I own, so the cards are not faulty.

I have heard the board will work with a GTX 6XX series if only one CPU socket is occupied, but not when both are. It also has problems with the latest Firmware. It will also work if both PCIe slot 1 & 3 are occupied in SLI mode. I have also now heard that other owners of this motherboard can't POST with ATI Radeon 79XX series GPUs either. Given the massive number of people with this problem, that would mean a ridiculous amount of boards have to be RMA'd.

Please note this is a BIOS POST Firmware issue, the board isn't faulty and neither are the cards. This has been replicated on MANY ASUS Z9PE-D8 WS motherboards, not just mine. The forum threads that back this statement up are as follows;

http://www.overclock.net/t/1261060/should-we-have-an-official-asus-z9pe-d8-owners-thread/80

http://rog.asus.com/forum/showthread.php?27225-ASUS-Z9PE-D8-WS-Issues-detecting-Quadro-K5000-amp-GTX-6XX-series-GPUs-%28Q-Code-62%29&country=&status=

http://vip.asus.com/forum/topic.aspx?board_id=1&model=Z9PE-D8+WS&SLanguage=en-us

http://www.newegg.com/Product/Product.aspx?Item=N82E16813131817 - see the reviews for information

http://www.newegg.com/Product/Product.aspx?Item=N82E16814133468

The problem still exists and really does need to be given SERIOUS consideration or definitive details on how you have made these cards work. Please note, if testing again could you please try with a Quadro K5000 rather than a card I do not have, as demonstrating the GTX 680 does not help me.

2013/1/2 Cherry Xie

ASUS CASEID=RWTM201212291220234472-671 Please click here if you wish to reply this mail!
Dear Valued Customer,

Thank you for contacting ASUS Technical Service.

Do you have a low end vga card to test on the motherboard?
We just use GTX 680 to test on the motherboard and it works normally.
You could refer to attached pictures.

You'd better contact with your retailer to check your motherboard. If it is due to motherboard faulty, your retailer would send the motherboard to us for RMA (Return Material Authorization) service.

Sorry for the trouble. Wish you a good day.

If you continue to experience issues in the future, please do not hesitate to contact us again.
Best Regards,
Cherry
ASUS Global Technical Support Center

Original Message

From :
Sent : 2012-12-29 4:20:30
To : "[email protected]"
Subject : Re:Re: Motherboard Z9PE-D8 WS [ID=RWTM201212281945496926-336]

[CASEID=WTM201212291220234472]

Hi, I have cleared the CMOS and this did not work sorry. As the graphics card is very
new, there are no firmware updates available as of yet (this has been confirmed by
PNY). Alot of similar issues are being discussed in the Asus forums -
http://vip.asus.com/forum/topic.aspx?board_id=1&model=Z9PE-D8%
20WS&SLanguage=en-us

I have tried also enabling the VGA palette snoop, tried various PCIe slots, forced
offboard VGA and the only thing that works is resetting the system multiple times
before it gets past the BIOS POST perhaps 1 in 3-4 times.

I have found this issue is not just with the Quadro K5000, but also with the nVidia GTX
670, GTX 680, GTX 690 (PLX issue) and various ATI high end graphics cards. It
appears to be a problem with the ASUS Z9PE-D8 WS firmware itself.

Original Message

From:ASUS TSD
To:
Date:2012-12-29 03:06:34

Dear Valued Customer,

Thank you for contacting ASUS Technical Service.

Please clear CMOS for a try.
Also, please keep latest firmware of your vga card.

Sorry for the trouble. Wish you a good day.

If you continue to experience issues in the future, please do not hesitate to contact us
again.
Best Regards,
Cherry
ASUS Global Technical Support Center

Original Message

From :
Sent : 2012-12-28 11:46:49
To : "[email protected]"
Subject : Motherboard Z9PE-D8 WS
[CASEID=WTM201212281945496926]
Apply date : 12/28/2012 11:45:49 AM(UTC Time)

[Contact Information]
*Name : David
*Email Address :
Phone Number :
City : Sydney
*Country : Australia

[Product Information]
*Product Type : Motherboard
*Product Model : Z9PE-D8 WS
*Product S/N : C8S1AG000XXX
Place of Purchase : www.nextwarehouse.com
*Date of Purchase : 2012/12/11

[Motherboard Specification]
*Motherboard Revision : Not Applicable
*Motherboard BIOS Revision : 3302

[VGA Card Specification]
*VGA Card Vendor : PNY
*VGA Card Model : Quadro K5000
*VGA Card Chipset : Kepler GK104
*VGA Card Driver : 310.70 & 307.45

[CPU Specification]
*CPU Vendor : Intel
*CPU Type : Dual Xeon 8 Core Socket 2011
*CPU Speed : 3.0GHz

[Memory Specification]
*Memory Vendor : Corsair
*Memory Model : CMZ64GX3M8A1600C9
*Memory Capacity : 64GB (8x8)

[HDD Specification]
HDD Vendor : Intel
HDD Model : 520
HDD Capacity : 480GB

[Add-on Card Specificatio]
Add-on Card Vendor :
Add-on Card Type :
Add-on Card Model :

*Operating System : Windows 7 64bit

[Problem Description]
nVidia PNY Quadro K5000 does not boot (Q-Code 62) when card is inserted into PCIe
slot 1. The mobo will get past Q-Code 62 if the card is inserted in PCIe slot 3 (but not
7), however hangs at Q-Code 62 with multiple speaker "beeps" 4/5 times. The mobo
boots with the Quadro K5000 sporadically. I have tested the following configurations;

1. Tried the Quadro K5000 card in the Asus P8P67 LE and it works fine.
2. Installed an EVGA GTX 480 in the Z9PE-D8 WS mobo PCIe slots 1 & 3 - both work
fine.
3. Updated Z9PE-D8 WS firmware to version 3302. Did not fix problem.
4. Tested with both nVidia driver 307.45 and 310.70. Did not fix problem.

Note; No.4 was a long shot as the issue occurs during BIOS POST, well before the
Operating System boots. It appears once past Q-Code checks, there are no issues. It
would seem the Firmware does not know how to handle the graphics card. I have
spoken to PNY about the issue and would be happy to send the details of that
conversation as well.

Regards,
Dave


----------



## Morachi

Just an update guys, i've been in touch with PNY and had the following response;

From: Cecile Tom tom
Date: Thu, Jan 3, 2013 at 2:38 AM
Subject: RE: RMA Request via Web - Quadro Graphics Cards#assistance 1252T027
To: David

Hi,

Sorry but we don't have a list with compatible motherboards otherwise i have forwarded this mail to the person which are in charge of tests before to be in the market.

Regards an happy new year.
Cécile
................................

So I really don't know how to read this, as it might mean Cecile has forwarded this, or they can't because there isn't any testing before distributing to market? Either way i've responded.

I have also been contacted by nVidia and have a teleconference with them tomorrow regarding the issues i've been having (nice of them to call from the USA to Australia I thought - excellent customer service). I'm hoping their support team can help me more than PNY and ASUS have thusfar.

Richard Swinburne from the ASUS ROG thread has responded with this;

"Hi,
Thanks for the detailed analysis and troubleshooting. I have passed this thread on and hopefully will hear back soon from our product manager or RD team. "

So that sounds more promising as the level two tech team in Taiwan have done testing on only one board and by the sounds of it, just the once - getting lucky then calling it a day. Fingers crossed these guys (ASUS/nVidia) can between them, come up with a solution.

I'm strongly considering compiling a list of service request numbers logged so we can get a count of just how many people have logged this complaint (to validate it isn't just a single corrupt motherboard as has been suggested!).


----------



## Morachi

Jackpot! Just as I dismiss PNY support their engineers come back with this gem (the reason for the problem - guess what, its a motherboard related issue in supporting the new cards as suspected);

Hi Dave,
*
which BIOS are you using on the mainboard. From another customer I know the K5000 is working with BIOS 0703 and 3109 for the Z9PE-D8-DW board. According to Nvidia the reason for this behavior is the 256mb memory BAR BIOS of the K5000 which has to work in the mainboards. So a fix must come from the mainboard vendors.
*
*
Mit freundlichen Grüßen/Best Regards
Ihr/Your PNY Support Team
..........

Ive requested further information and specifics on the identified issue also requesting the contact details so I can put them in contact with ASUS to collaborate on a fix. It may be premature but this is the best news all week!!!

*Update (found this interesting about PCIe base address registers - BARs) - download.intel.com/design/intarch/papers/321090.pdf


----------



## Morachi

Here is an update on what works and doesn't with the Quadro K5000;

Please note, all of the below is with the 3302 mobo BIOS Firmware.

1. Will not work in PCIe slot 1
2. Works 1 in 10 times or so in PCIe slot 2
3. Works 1 in 3-4 times in PCIe slot 3
4. Works 1 in 3-4 times in PCIe slot 4
5. When plugged into PCIe slot 1 and onboard VGA is enabled, I can remote (VNC) to the machine (as no display output occurs to the K5000) - This means the card is detected (perhaps?) however the display output defaults to the onboard VGA.

When the card does work the Q-Code cycle is as follows;

1. Q-Code 62 - single beep (a second or two just before this code shows, the ASUS splash screen can be seen - it indicates a successful display detection)
2. Q-Code AE
3. Q-Code 22 (just as the OS loads, it remains on this code for the duration of the HPC being powered on)

When the card DOES NOT work, the Q-Code cycles is as follows;

1. Q-Code 62 - single beep followed by 4-5 successive beeps (at this point if the ASUS splash screen hasn't shown, then it is certain the display won't occur)
2. Q-Code A2 - single beep followed by 8 successive rapid beeps (the machine sits at Q-Code A2 after this)

Troubleshooting steps i've taken so far are as follows;

1. Removed RAM (to one stick of 8GB)
2. Reseated RAM
3. PCIe slot reallocation
4. Disabled onboard VGA via jumper switch
5. Flashed BIOS multiple times (update stepping)
6. Cleared CMOS multiple times
7. Tried two GTX 480 GPUS (ASUS, EVGA) - Both worked in PCIe slots 1, 2 and 3 every time)
8. Enabled VGA Pallette Snoop
9. Disabled all PCIe slots except PCIe slot in use (to see if a conflict was occurring)
10. Swore at the HPC with every four letter word I have in my vocabulary, then had a coffee
11. Tested Quadro K5000 in other PC's - each single socket board booted successfully

The motherboard and graphics card have both been proven to have no issues with the hardware itself (testing the cards successfully in other boards etc).

PNY are aware of the issue, however it is out of their hands.

nVidia gave the details of the root cause to PNY (who passed it on to me), they may have a firmware in the future that assists

ASUS at this point still maintain they can get a GTX 680 working in the machine, booting first time, every time with their own branded GPU.

At this point I am still awaiting a call from nVidia to discuss the matter further and have requested the contact details of the person who gave the 256MB BAR details to PNY (The BAR issue is related to the Intel C602 chipset from my brief understanding and research this morning). I hope to pass these details (and the contact information is cleared by nVidia) to ASUS so they can consider resolving the problem in their next BIOS Firmware revision.

Other user responses;

1. A lad who posted on newegg.com was able to get the K5000 working in PCIe slot 2 - details below;

The motherboard started out with the 0703 Bios upgraded to 3109, then to 3206. Random boot issues with the K5000 (in slot 1) manifested after all updates. At this time, I figured the PCIE slot 2 configuration where things were working for me. I have since updated to 3302 with no apparent difference, the K5000 still boots unreliably in slot 1. *Note; This config works with a single CPU occupied and the end user has not upgraded to a second CPU yet until they are clear the issue with the BIOS is resolved.

2. A user on the ASUS ROG forums was not able to get his ATi Radeon 7950 to work in any PCIe slot except slot 3. He also tried an ASUS GTX 650 which also would not work;

I'm having the exact same problem. I've tested a couple of cards (RADEON Hd 5450, nvidia fx3500 in sli) and they work fine. However I just bought the hd 7950 by xfx and get the same error as you do. It seems the motherboard doesn't except the latest graphics cards even though it supports pci-e 3.0.

I''ve updated my bios to the latest version and still nothing. I forgot to mention also I tested an Asus 650 graphics card as well. That still didn't work even though it was the same make as the board. *Note; This end user now has the Radeon 7950 working in PCIe slot 3, booting every time, however by the sounds of it still can't get PCIe slot 1 to work with it.

3. ASUS Taiwan support have the GTX 680 working in PCIe slot 1 with the 3302 motherboard firmware stating it boots every time.


----------



## Morachi

Time for an update!

An engineer from PNY Germany contacted me with some very interesting news (as he was aware of the issue). Essentially at BIOS POST allocation of resources occur to the PCI memory BAR, and by what I gather this happens in a dynamic/rando fashion - which is why the card will POST sometimes, but not all the time. Here is the document detailing the how/why etc - http://msdn.microsoft.com/en-us/library/windows/hardware/gg462986.aspx

The PNY technician gave me a test firmware to drop the memory BAR request from 256mb to 128mb in an effort to prove the hardware was not faulty (neither the motherboard or the GPU) and that the BIOS code needed some serious looking at - please note I can not distribute the K5000 test firmware as it is not a FIX, just a means of how to validate where the problem stems from.

nVidia cancelled the phone call upon hearing that I had been in touch with PNY, and after detailing the issue they then verified they knew about it. I have asked for a contact there so I can put both the ASUS and nVidia/PNY technicians in touch to expedite a new firmware release. I hope to be kept in the loop on this, perhaps even being able to test the new "test" BIOS myself to validate it works here (since I own all the components in question).

My next big concern is the Tesla K20 GPGPU and how it will operate when it arrives end of this coming week. I will be flashing the K5000 back to its original production BIOS and testing the K5000 + K20 in their intended slots (PCIe 1 & 3) prepping for the Maximus 2.0 rendering platform.

IMPORTANT!: I've also been told by ASUS R&D that the Q-Code 62 is actually "b2". The difference between the two errors is as follows below;

Event code 'B2" = Legacy Option ROM Initialisation
Event code '62" = Installation of PCH Runtime Services

I am actually more inclined to believe it is now B2, simply due to the fact PCH runtime services are more about SSD/HDD issues than POST display initialisation.

I may have to see if turning off the legacy ROM support makes any difference early next week when I get a copy of the production BIOS for the K5000 back from PNY.


----------



## adamfilip

Ive emailed PNY Support the following

"Hello I am interested in using the PNY K5000 Quadro in a Asus Z9Pe-d8 or Evga SR-X motherboard I have read these motherboards are incompatible with the k5000

can you suggest a dual LGA 2011 workstation motherboard that is compatible?"

From PNY Support

"Actually, it is reported that our PNY Nvidia K5000 works in the Z9PE-D8-DW with versions 0703 and 3109 of the motherboard BIOS installed. Otherwise, a recently manufactured SuperMicro Xeon-based workstation-class motherboard is likely to work very well. I hope this helps."


----------



## Morachi

Hi Adam,

I had the same response as well, unfortunately in my case, this did not turn out to be true. As the R&D PNY tech pointed out, this is an issue with BIOS firmware that doesn't allocate the requested 256mb Memory BAR resources at boot, resulting in the GPU failing initialisation.

ASUS are aware of this and currently working on a BIOS fix to correct PCIe resource BAR allocation at boot. I'm still working with the PNY Germany R&D office and ASUS on this very issue.

I'll update when a full fix is getting closer to the finish line.


----------



## franklins

Another z9 owner here with woes of his own!.
Key Specs.

Z9 PE-D8 WS (obviously!)
2x E5-2690 CPUs
16gb of memory (1333mhz 9-9-9-24)
3x GTX 680s (Zotac models all purchased at the same time, factory BIOS 2GB memory per card)
3x 27inch dell u2711 monitors.

To be brief, I'm using the 3 680s in slots 3, 5 and 7, BIOS 3302 and all settings at their defaults. Boot problems haven't been quite as severe as others here but about 2/3 times out of 10 the system will fail to post and halt at code "94", though occasionally, code "62/b2" is where it freezes, which seems to be the more common in general.

As others have experienced, if I'm able to boot into the OS (windows 7 x64), all seems very stable. However, I am encountering another very strange issue which concerns me. For some reason, I cannot enable Nvidia surround when one of each of my 3 monitors is connected to each individual card (I have each screen connected to the card's primary DVI connector as advised).

Needless to say I have all the latest MB chipset drivers installed and for the 680s, I'm using 310.90 (the latest). However, when I open the Nvidia control panel and select "span displays with surround" and then click "configure", I get a box like the one below......

.

That's it...completely blank with the apply button greyed out.

This also occurs when using just two cards in slots 1 and 5, 3 and 5, 1 and 7 and 3 and 7 (with 2 displays connected to one card and the third connected to the other).

However, if I use two cards in either slots 1 and 3 or 5 and 7, (again with 2 displays connected to one card and the third connected to the other) surround works as its supposed to. So essentially, it would appear to be an issue related to the MB's chipset and/or which CPU is responsible for which PCI-E slots.

As I understand on this board, each CPU is assigned 2 of the 4 blue slots (slots 1 and 3 for CPU 1 and slots 5 and 7 for CPU 2) and for surround to work properly, the cards can't be in slots that are handled by different CPUs which means that a 3 card, 3 monitor setup with one monitor per card is impossible as the third card is always assigned the other CPU .

I have done several clean installs of both windows and the video card drivers and as yes, nothing has resolved this issue.

The odd thing is that 3x SLI on a single screen works fine and I can activate all screens at the same time regardless of which slots are populated or if there are 2/3 cards installed.

Anyone experience similar or have any theories or suggestions?


----------



## tomyboy

Quote:


> Originally Posted by *BlackenedTush*
> 
> I am using 64GB of Corsair Vengeance DDR3 1600. It only uses 1.5 volts. This RAM is on the ASUS Qualified Vendor List (QVL), .


I can find this kit on QVL?
My motherboard and other components are still waiting in the store, because of the RAM not being delivered.
First I orderd 4 Kingston kits 2x8GB. When I read this thread yesterday, I want an canceled it. I order the CORSAIR XMS 64GB (8x8GB) instead.
http://www.newegg.com/Product/Product.aspx?Item=N82E16820233308
Anyone using this configuration? Hope it is going to work?


----------



## Morachi

Tomyboy;

I am using the Corsair Vengeance 64GB 1866 8x8 kit which works flawlessly. I have been told that the techs at ASUS couldn't get the Quadro K5000 working with this kit, however they could with 32gb of registered memory (oddly enough multiple other people have failed to do so even with registered RAM). Frankly I think they got lucky, not to mention they didn't try the card in PCIe slot 3 (which the GPUs in question have been able to boot 1 in 3-4 times successfully).

BIOS rollbacks are unsuccessful either, all stated SBIOS updates (that are claimed to have worked) have been disappointing to say the least. This includes the test 3402 BIOS I have obtained from ASUS. I'd love to know what they did actually change, however haven't heard back so far.

Personally i'd say stay away from this motherboard until the SBIOS is updated, if you can go for another C602 based motherboard, do so - ASUS dropped the ball on this one compatibility wise.

So far to date the following GPUs (and vendors) have had problems getting the card to work, all exhibiting the exact same problem (error at Q-Code b2/62, 5 short beeps, followed by 8 rapid beeps, no display followed by a system hang).

nVidia Quadro K5000 (PNY)
nVidia GTX 690/680/670 (ASUS/eVGA/Gigabyte)
ATI Radeon 79XX series (can't recall the vendors)


----------



## tomyboy

Thank you for the heads up, but the motherboard is already paid for, so that's it.








About the ram I hope there will be no problem, i will do the mem test first.
Should I update the bios immediately, to let say 3109, I will be using Windows 8 Pro, or do I just leave it alone?
About the graphic card, I was looking at K5000 awesome ....( just looking , but really shame that there is a problem with this motherboard.
Personally, for my projects, I only need raw processor power and little more ram. I am using old Quadro 3800, planning to update to new K4000 when it comes out, but we will see if they fix the problem.
Not doing some intense GPU applications, 3ds Max doesn't give any benefit when rendering (only Vray RT), Premiere Pro also.
I only saw the big difference when using ELEMENT 3D in After Effect. My gaming rig with GTX 690 is 3x faster when rendering than Quadro 3800. This is the only app on my system that is totally using GPU for rendering (3D Ray Trace).


----------



## lloyd mcclendon

Well, my single E5 2687-W finally arrived in the mail today. And i'm pretty shocked, but it was seriously turn-key. I've never had this good of luck with any build before - swapped out my old i7-950 and p6x58d-e without one single problem. I turned the power button on, didn't even go into the bios, and was back at my desktop with 4 more cores but otherwise like I never left. I expected at least one hiccup... especially considering the contents of this thread. The only thing I forgot was I did not compile the different NIC module before shutting off with the old board.

I think the majority of the problems with the board are due to 1) the need to use windows







2) improper assembly (extra standoffs anyone?







) 3) the gtx600 series seems to be quirky with it, but should work once the BIOS has matured a bit. I seriously doubt there are 1/4 as many DOA boards as people claim. Case in point, I bought this board open box, which means someone had it, and either said it's dead, and it wasn't, or maybe it just wasn't going to work out for them.

For now I have quite a few air bubbles floating around, but now that the proof of concept has gone very well (long term stability to be determined, coming off of 97 days uptime), I will probably spring for the second 2687-W in 4-10 weeks, and during that downtime period also overhaul the loop considerably.

I am also looking at the waterblock for the board here: http://www.aquatuning.us/product_info.php/info/p14170_EK-Water-Blocks-EK-FB-KIT-ASUS-Z9PE-D8---Acetal---Nickel.html Then really the only thing left is to replace the 2x 570s with 4x new cards sometime next year, and I'm about done forever.







Now to remove the SLI bridge and see if I can pass one to a VM...











What kind of temp deltas across the cores are you guys experiencing? Between cores 3 and 7 i'm looking at a 10-12 degree difference, which is a lot, but I know often times with intel chips such a spread is normal.. the 950 was 6-8 degrees..


----------



## Morachi

Hi Lloyd,

Glad to see you are happy with your new build. You're quite lucky to be running 570's as the issue is with newer GPUs at present. Windows isn't the issue here either as the problems that have been exhibited are at boot, well before any OS is loaded. nVidia are now working with ASUS to assist in a fix.

EVGA have come out with a hack for their branded GTX 680 in the meantime until ASUS sort out the SBIOS - From evga : "We basically removed the boards option to set what gen the card runs at and made it fixed. The motherboard is supposed to do this, however since it was not doing so correctly, we removed its option and hardcoded the card to work around your motherboards shortcoming. So this should work well for you now.".

I have a copy of the 3402 beta SBIOS update, which sadly doesn't fix anything, if anything it makes matters worse for certain types of RAM, forcing 1333 speeds. Suspect XMP issues there.

Time will tell, until then i've had to put my Tesla K20 on the market as it was causing spontaneous reboots every few days - so there is ANOTHER issue to be looked at, which I simply don't have the time for.


----------



## lloyd mcclendon

I'm starting to exhaustively dig through the bios ... this is great

under advanced, chipset, memory

there's an item to set your *ALTITUDE FROM SEA LEVEL* in meters









Does any other board have that?







.. I don't know the science behind how altitude can affect RAM, but it probably does.







I found my altitude here http://www.daftlogic.com/sandbox-google-maps-find-altitude.htm and unfortunately I am right about at the default 300M. Some of you may have to change that and could pick up a full jiggahert or two


----------



## tomyboy

Just finished my rig.


I would like to use two SSD in raid 0. First I connected them on Blue Intel SATA3, created radi 0 with SLI Software RAID Configuration Utility, no problem so far.
When I tried to install Windows 8 and load SLI Raid driver for Win8, I got message that driver is not compatible x64 or something like that. tried x64 and x86 , nothing.

Intel rapid Storage Technology enterprise Option ROM utility allso doesnt work.
There is no option when to press control+I when booting, to go into the utility.
Is there something to enable in bios other than sata mode to RAID?

Marvel is allso not working.
Created Raid 0 successfully. WIndows 8 even recognize virtual drive, no drivers needed.
But when I tray to create new volume the procedure stars and doesnt end.

What am I doing wrong, really starting to give up on the raid setup?
Thanks for any advice.


----------



## Sweeper101

What kind of systems are you using this board for?


----------



## lloyd mcclendon

just the usual, web browsing, email, solitaire, etc.







I think most people use it for video / photo editing / rendering. I myself run a cluster of VMs for some complex software projects, 1000+ threads hammering at databases etc etc. It would be nice if I could get the project to build again









RE the raid setup, why anyone would use hardware raid is beyond me. The fact is that you are coupling your dearly important data to that particular raid controller. When your board inevitably fails, you had better hope you can find the same controller, and that the company that made it is still in business. Software raid is the way to go IMO. mdadm. If you don't have a spare controller, you're taking a huge risk, and if you care enough to raid the data in the first place, it's probably too big of a risk.

http://backdrift.org/hardware-vs-software-raid-in-the-real-world-2

"On-Disk meta data can make it near impossible to recover data without a compatible RAID card - If your controller goes casters-up you'll have to find a compatible model to replace it with, your disks won't be useful without the controller. This is especially bad if working with a discontinued model that has failed after years of operation"

given what I have experienced with marvell, and what I've read others experiencing, I wouldn't be surprised if they weren't around 10 years from now. So you setup the marvell raid, use the board happily for say 8 years. One day the board goes, marvell no longer exists, you can't find a replacement. Now what? I hope your backup file system was under a different raid controller







or 8 years of work down the toilet


----------



## tomyboy

I just wanted to use two SSD in Raid 0 for the system, just for better felling.








I am used to speeds up to 1500 MB/s with revo drive 3 X2 on my another gaming machine. I thought I would use Raid not Revo, because of all the obvious problems you have with graphic cards and multiple pcie slots occupied.
I don see any problems even it crashes. I have complete system and software up and running in two days.
Butt I guess one SSD for the system will have to do...


----------



## BlackenedTush

lloyd,

I see that "it's beyond you why anyone would use a hardware RAID..."

I've read your posts and you seem to be intelligent and quite experienced; but to me your strong conclusion about RAID is a bit puzzling. You're seeing it as kinda B&W thing where a lot of other intelligent people would be seeing it as gray. The idea that it's _"...beyond me"_ would suggest to me that you are either 1) admitting that you are not as expert as I would have surmised from your posts, or 2) that you consider your knowledge as definitive and above the opposing and studied opinions that I have read in my own research on the subject.

It seems to me tho that on a very basic level you're promoting the principle that putting all of your eggs in one basket is not wise. I would heartily agree with you on that point.

The way I see it, when one is building a storage strategy, one must set out to to strike the most appropriate balance between performance and risk.... and that strategy is out of necessity qualified by one's budget. But not only that; but it is also qualified by one's "knowledge" of the choices. From my view, that "knowledge" factor is far more nebulous than we'd ever want to believe. I say that because the stuff that forms this knowledge is largely based on our impressions and a mixture of heresay, 2nd-hand data, and the perceptions, sales propaganda, prideful bravado, and assumptions of others (which we assign credibility to). And for the most of what we learn in this world, we're dealing with a large amount of questionable input.

I looked at the same Marvell company that you did. And my "knowledge" of them has been largely formed from my impression of their company and their products based on forums and reading other internet babble about peoples' Marvell issues. I would imagine that your own impression of them and their products is second hand as well. You see a very positive view of the quality of their products, their longevity as an on going entity, and the immutability of their programming strategy going forward. And it seems you assume that such a foundation ensures a predictable support environment for the future of a current Marvell software RAID configuration. Yet I look and see something different. I've read enough negative things about their chipset being problematic on our particular board that I was wary when configuring my own system. And indeed in my initial testing I did have my own issues with the chipset. Now I had no way of predicting or verifying what their support structure will be in the future toward addressing a RAID failure- I could only assume. I only know that I was only too happy to avoid my troublesome Marvell subsystem by using only the Intel chipset and to employ a hardware RAID controller the my research justified buying. Certainly there were many other factors that I considered. Rendering efficiency for large video composites was one of the primary factors. And after a almost a year in service, I am happy with the performance I've experienced. And so far it has been reliable. And of course I'm hopeful that my controller card is quality enough to never be a problem. But we can say that about each and every piece of HW and SW (and their synergy) we've spec'd for our builds. There are numerous potential failure modes now and in the future including new OS, drivers and incompatible hardware we bring into the mix. And users of this particular complicated MoBo have varied experiences with it. And we haven't even talked about the innumerable enemies of our data coming from software/hardware corruption, virus attack, acts of God, using large/dense consumer HDDs, etc...

In any case, I made sure that my own storage strategy included redundancy beyond the meager amount that any RAID setup offers - it includes rotating two hot-swappable BU disks, and offsite. That is a reasonable approach considering what I think I know, my budget, the relative importance of the data, etc..

I believe that the RAID approach anyone takes, whether hardware or software, is justified by the application. There are pros and cons for each and one's own application determines their particular weight. But risk always needs to be mitigated. Nothing is perfect; but we should do the best we can. In my view, reconstructing a RAID array should not be the primary strategy for dealing with a controller failure (just a last resort). So whichever is adequate or not at reconstruction is really beside the point. My personal recommendation would be to rely on one (or more) backups of our data on at least one quality disk. And that approach is valid whether prime storage is HW RAID, SW RAID or a more simple drive setup.

Thank you for your input on this board. Cheers.

BlackenedTush


----------



## Morachi

Hi guys,

Just thought i'd relink the Asus ROG thread as it has been more active than this one lately. http://rog.asus.com/forum/showthread.php?27225-ASUS-Z9PE-D8-WS-Issues-detecting-Quadro-K5000-amp-GTX-6XX-series-GPUs-%28Q-Code-62%29&p=220802&viewfull=1#post220802

Summary as follows;

ASUS acknowledged the following -

They can't get the Quadro K5000 working with unregistered RAM (then again we can't get it working with either reg or non-reg RAM)
There are issues with GTX 6XX series GPUs not of their making (EVGA/MSI/etc). Suspect it is how they've programmed their VBIOS to lock the PCIe gen.
Suggested that for 690's, the PLX would need to be flashed (which has been a known fix for a LONG time now).

Note; no response has been provided regarding just what the 3402 beta SBIOS is meant to do, still waiting on this.

EVGA commented on the following -

They have a fix for the GTX 680, this fix locks the Gen that PCIe runs at (3.0 in this case) which works around the SBIOS limitations. EVGA will give out the PLX ROM update to those who ask, not requiring the card to be sent back.

nVidia -

Stated they are working with ASUS to resolve the issues.

On a personal note, I have also noted the Tesla K20 suffers from a spontaneous reboot problem once a day. I suspect this to be a hardware conflict with the device driver - much like the Corsair I series watercoolers and their software. I've removed the K20 (actually selling it for the time being) as having a device which crashes the machine only compounds the problems i'm seeking to have resolved, I can always come back to this later. The K5000 is still suffering from detection - i've removed it and installed the GTX 480 for the moment as none of the programs used for rendering presently take advantage of the GPU.


----------



## BlackenedTush

Very Impressive looking rig tomboy.
I love that Cosmos case and the clean lines it promotes. Great job!


----------



## lloyd mcclendon

Quote:


> Originally Posted by *BlackenedTush*
> 
> lloyd,
> 
> I see that "it's beyond you why anyone would use a hardware RAID..."
> 
> I've read your posts and you seem to be intelligent and quite experienced; but to me your strong conclusion about RAID is a bit puzzling. You're seeing it as kinda B&W thing where a lot of other intelligent people would be seeing it as gray. The idea that it's _"...beyond me"_ would suggest to me that you are either 1) admitting that you are not as expert as I would have surmised from your posts, or 2) that you consider your knowledge as definitive and above the opposing and studied opinions that I have read in my own research on the subject.
> 
> It seems to me tho that on a very basic level you're promoting the principle that putting all of your eggs in one basket is not wise. I would heartily agree with you on that point.
> 
> The way I see it, when one is building a storage strategy, one must set out to to strike the most appropriate balance between performance and risk.... and that strategy is out of necessity qualified by one's budget. But not only that; but it is also qualified by one's "knowledge" of the choices. From my view, that "knowledge" factor is far more nebulous than we'd ever want to believe. I say that because the stuff that forms this knowledge is largely based on our impressions and a mixture of heresay, 2nd-hand data, and the perceptions, sales propaganda, prideful bravado, and assumptions of others (which we assign credibility to). And for the most of what we learn in this world, we're dealing with a large amount of questionable input.
> 
> I looked at the same Marvell company that you did. And my "knowledge" of them has been largely formed from my impression of their company and their products based on forums and reading other internet babble about peoples' Marvell issues. I would imagine that your own impression of them and their products is second hand as well. You see a very positive view of the quality of their products, their longevity as an on going entity, and the immutability of their programming strategy going forward. And it seems you assume that such a foundation ensures a predictable support environment for the future of a current Marvell software RAID configuration. Yet I look and see something different. I've read enough negative things about their chipset being problematic on our particular board that I was wary when configuring my own system. And indeed in my initial testing I did have my own issues with the chipset. Now I had no way of predicting or verifying what their support structure will be in the future toward addressing a RAID failure- I could only assume. I only know that I was only too happy to avoid my troublesome Marvell subsystem by using only the Intel chipset and to employ a hardware RAID controller the my research justified buying. Certainly there were many other factors that I considered. Rendering efficiency for large video composites was one of the primary factors. And after a almost a year in service, I am happy with the performance I've experienced. And so far it has been reliable. And of course I'm hopeful that my controller card is quality enough to never be a problem. But we can say that about each and every piece of HW and SW (and their synergy) we've spec'd for our builds. There are numerous potential failure modes now and in the future including new OS, drivers and incompatible hardware we bring into the mix. And users of this particular complicated MoBo have varied experiences with it. And we haven't even talked about the innumerable enemies of our data coming from software/hardware corruption, virus attack, acts of God, using large/dense consumer HDDs, etc...
> 
> In any case, I made sure that my own storage strategy included redundancy beyond the meager amount that any RAID setup offers - it includes rotating two hot-swappable BU disks, and offsite. That is a reasonable approach considering what I think I know, my budget, the relative importance of the data, etc..
> 
> I believe that the RAID approach anyone takes, whether hardware or software, is justified by the application. There are pros and cons for each and one's own application determines their particular weight. But risk always needs to be mitigated. Nothing is perfect; but we should do the best we can. In my view, reconstructing a RAID array should not be the primary strategy for dealing with a controller failure (just a last resort). So whichever is adequate or not at reconstruction is really beside the point. My personal recommendation would be to rely on one (or more) backups of our data on at least one quality disk. And that approach is valid whether prime storage is HW RAID, SW RAID or a more simple drive setup.
> 
> Thank you for your input on this board. Cheers.
> 
> BlackenedTush


----------



## BlackenedTush

lloyd,










BlackenedTush


----------



## ikem

idk if i have posted here....


----------



## anubis1127

Looks nice. I only wish I could have gotten mine working.


----------



## lloyd mcclendon

but seriously though, if you're doing hardware raid to ensure your data stays alive in the event of component failure, e.g. raid1/raid10, this doesn't really buy you very much as it simply moves the single point of failure over to the controller, which can and will fail on you.

http://augmentedtrader.wordpress.com/2012/05/13/10-things-raid/

apart from a raid0 setup where you really don't care about redundancy / saving the data at all, I can't picture any situation where hardware raid is the smarter way to go. If you insist on doing it, you had better well have a spare controller [behind glass and climate controlled and atomic bomb proof].









And yes, Marvell sucks, all of the chips I've owned have had fun compatibility problems...


----------



## BlackenedTush

Morachi,

I'd really love to get my hands on a new Maximus 2.0 combo, but alas I am a poor student. But I hope to continue to hear of your progress on the compatibility fight. Thanks for your your postings on the GTX and K5000 struggle.

When I was speccing out my Z9PE-D8 rig a year ago, I had established a budget of $6k US and settled on a Gaming card, the GeForce GTX670. But Maximus 2.0 represents the Ferrari of workstations. If you got the bucks and you wanna rock, you buy it. But is it it worth the price you'd pay over a much cheaper corvette? I don't have the luxury to be able to say, or to be able to demand the best.

I had looked hard at the relative roles that CPU, RAM, disk cache, & GPU played in the preview & rendering processes of Adobe's CS5 and CS5.5 apps. CS6 was on the distant horizon. I looked at Autodesk's recommendations for 3DsMax. Adobe in particular has published a lot of Info on setting up the various apps within the suite to milk the best performance out of whatever configuration you might have. They discuss at length the role of CUDA, etc., in Premier Pro's "Mercury Engine." I love that. It was very helpful. My budget was only $6k. I came up with several rigs that varied from being CPU/RAM heavy to GPU heavy. Knowing that future Adobe offerings would make better use of cores through more efficient threading I settled on a balanced compromise spec: two 6-core Xeons, 64GB RAM, and a Fermie based Quadro 2000 (but I wanted a Quadro 5000).

But at the time Kepler cards were coming out on the gaming side. WS cards seemed like dinosaurs. And so I had been trying to compare the relative value of a workstation card vs a gaming card. And I had read about the new Maximus technology. But it seemed to me at the time that maybe NVIDIA was just as savvy at business as they were in GPUs. I surmised that as the Kepler technology was coming in, NVIDIA was looking for ways to remove some obsolescence from the older Tesla & Quadro cards to give them new life in the market. Maximus could certainly do that. But it seemed to me that the cost of a workstation card was very very high in comparison to the benefits it advertised. And a new Maximus pairing was costing much much more than that for not that much more of a gain. Yes, it was intriging to me to consider that I might build a Maximus config with some used cards; but that represented a lot of effort. And 10-bit color support on the higher end Quadros like the 5000 would require an equally high-end (high-priced) graphics monitor that wasn't in my budget either. As I looked at the gaming cards more deeply, I concluded that whatever I gave up in Professional appeal, high-end user support, advertised quality/stability, 10-bit color support, and some other general but mysterious graphics-processing technologies, was small in comparison to what I would gain in Cuda cores, Direct X, video RAM, and other things that I knew Adobe apps would honor. AND at far lower cost... Plus, the number of users of gaming cards is much higher, so the quality and support one might expect from a GeForce would likely be somewhat comparable to that of the Quadro's. And I read blogs from video/graphics people who had great success with NVIDIA gaming cards. I decided to drop my Quadro 2000 plan and my Quadro 5000 dream. And I decided not to pursue any used Maximus config unless two compatible cards just happened to show up at my doorstep. I settled on an overclocked 4GB variant of the "new" GTX670. I knew there was a little risk because it wasn't on the tested list for Premiere Pro Mercury Engine processing; The 680 was eventually. But I knew the hack. And I also had seen postings on the ASUS board that some peeps had had trouble with 670 card on our board; but I also saw several posts of people overcoming the issue. ASUS was of little help as you know.

Yes I've certainly had my issues with this touchy MoBo and that includes getting the GTX670 to work in certain PCIe slots; but it works well now (in slot 5, BIOS v0503). And I have tried rendering in composites in After Effects in pure Open GL mode and it wasn't optimum. I found that a few of my composited items didn't appear in the output. But at least it was relatively fast, haha! Open GL is enabled in Photoshop and that works well. I need to explore what GPU only processing doesn't do well in AE when i have time. And I have had issues going from DirectX 9 to DirectX 10 in 3DsMax's viewport settings (while doing "camera per pixel mapping"); but I haven't determined that it's the card per se - I have the same issues at school with different workstations and cards. It may be 3DsMax. I am in some ways glad that I couldn't afford a Maximus solution considering the issues you are facing with this MoBo. But I really wish you the best toward them getting solved.

Beyond getting our MoBo to that point of awesomeness and stability we all dreamed it could be, one thing I'd really like to see from NVIDIA is a standardized benchmark for rendering that they would use for benchmarking their cards and publishing the results; but also, that they would make it available for download so potential buyers could clearly compare their current configs' performance against known Quadro or Maximus configs. Then we could better focus on what performance we might be realistically striving for as we give up one or two mac & cheese meals a week to save for it.


----------



## ABnormAL77

hi all

just got a z9pe-de ws but not working right









bios 3302
2x e5-2687w
8x8g ddr3 2400mhz
4xssd raid 0 for windows
4xevga 680 gtx 2g sc (beta 313.92 and all so tried 310.90)
2x 1500w ps (one for the evga cards and one for everythin else)
win 7 64 pro

some time the unit will not load , and some time the unit will BSOD with nvlddmkm.sys, some times it loads up with 3 680gtx and one will hava a yellow make on it with the device cannot find enough free resources (code 12)

this is driving me crazy

need help bad asus help is no good

need my pc to work every time i turn it on with 4 way sli working....

any one out there that can help????


----------



## tomyboy

I am relly disappointed so far and don't know what is going on?
Compared to the old sistem x980 with 12GB rama, this Xeon setup is in some application slower and not working OK.
On both systems I have the Win 8, on the same brand SSD, everything installed as it should be. On the first glance xeon is working great. But then, some aplicacion just doent work as they should.
For example looking at After Effect with same plugins instaled...
First I noticed that the AE sometimes takes 3x longer to start on Xeon. Than is the rendering with some projects(1080p DSLR footage with Looks filters) it took 10x longer than on x980 to render......








Even when the project files are loaded, the xeon system is taking its time. Even thou it is loading files from different SSD, both are on Intel Sata3 controller.
Really have no solution for this kind behavior compared to "inferior" old 1366 x980?
There is only big difference in cinebench, where the xeon is twice as fast.
And rendering test object in 3Dsmax 2013 is 20% faster with xeons, still not a big improvement, considering how much this setup cost..


----------



## YpsiNine

Hi,

I just bought this motherboard and I am only going to use the onboard graphics, but the bracket that has a cable with a connector on is nowhere to be found.

Anyone knows anything about this? Can't really find any info on the Asus site either.

Best,
X


----------



## Trotador

Quote:


> Originally Posted by *Xaurus*
> 
> Hi,
> 
> I just bought this motherboard and I am only going to use the onboard graphics, but the bracket that has a cable with a connector on is nowhere to be found.
> 
> Anyone knows anything about this? Can't really find any info on the Asus site either.
> 
> Best,
> X


Asus decided not to include it in such a pricey board. I got mine in ebay though have not installed it yet


----------



## Liron

Quote:


> Originally Posted by *Trotador*
> 
> Asus decided not to include it in such a pricey board. I got mine in ebay though have not installed it yet


That's strange. Mine came with it in the same bag as the serial port.


----------



## maratus

Hi All,

It may not be the best thread, but I didn't want to create a new one.

So far, there's an Intel C602-based board from Supermicro
http://www.supermicro.com/products/motherboard/Xeon/C600/X9DAE.cfm
called X9DAE, and it's the only their board with official 2-way SLI support. It also has 8 RAM slots / socket, provided you have enough courage to setup at least 256Gb to justify it









What do you think about this board? Does it have any chance to be more stable and have less compatibility problems than Asus? Is their customer support and bios updates regularity any better / worse? Their usual response is that they don't test boards with non-workstation class GPUs, so would it be a turnover for you if you want to stick with desktop card like GTX670?

And my latest question is more related to this board: Is it possible to run Z9PE-D8 with E5-1650 (single socket), ECC Registered ram (all 4 slots for CPU #1, 4x16 = 64Gb total) and 2x GTX670 in SLI mode? Does it make sense? I can go with P9X79 WS and 64Gb of ECC un-buffered RAM, but I don't want to risk as it's pushing IMC too far and I haven't even seen a matched 64Gb ECC kit yet...


----------



## Goudman

Have yet to assemble, but I have the Z9PE-D8 WS Motherboard (Rev # ? TBA, Bios Ver (will determine on bench testing soon)) with very similar components purchased: I.e. CPU: Xeon 2600 Family E5-2690 (2 Ea), Graphics: Asus GTX690-4G5D, Ram Corsair Vengeance 64GB (8 X 8GB) 2133Mhz CL 10. Due to my investments in the CPU's I am pretty sure I will stick to using the Z9PE-D8 WS board and start contacting Asus for them to provide the necessary Bios updates and or Firmware so that the Graphics will work seemlessly. I will eventually want this resolved since I plan to run the three 27" Samsung monitors I just installed. I will need ASUS to step up an take ownership to resolve these Graphics Card issues so I can run with my system build and have it actually work. How can such a high end motherboard be marketed with such little information or advanced notice informing the consumer all of the things that you can not attach to it? Do you believe ASUS will make everything right? If Yes, how long will this take?


----------



## maratus

You just have to send it to Asus, they do reflash their 690 cards and it's been working well AFAIK. May I ask you about your choice of RAM? Why did you get overclocked desktop kit instead of server-grade DDR3? Isn't that like the main point of Xeons that they support registered ECC memory?


----------



## lloyd mcclendon

i went through the same decision and went non-ECC... there is a performance hit with ECC as it calculates the parity. I can't remember the last time I had problems with good desktop modules, really the tech has come a long way in the past couple years (and it's all super cheap anyway). To me the benefits of ECC weren't that big of a deal ... it's not a production server.. whats the worst that could happen .. which will never happen anyway

The main benefit of the xeons as far as I saw it is obviously that you can run TWO due to the dual QPI link









IF only they would have unlocked the multiplier







i can't even imagine

I might be upgrading to the 600 series in the medium term, i'll call ASUS next week and see what they say about this whole mess. Somehow I managed to get transferred to one of their engineers last time which was very helpful..


----------



## BlackenedTush

lloyd,

Even tho it seems we apply the notions of invevitability and probability from different POVs, I again agree for the most part with what you are saying... I didn't purchase ECC RAM either; but not for the performance hit. I wasn't imagining the parity check duration being of much significance in the scheme of things. Same with DDR3 frequencies over 1600. My reason for purchasing consumer RAM was because the ECC is a lot pricier, and like you say, RAM tech has come a long way. But even so, "the worst that could happen .. which will never happen anyway..." is instability and very likely corruption of your OS, apps, and files And that stuff did happen to me. See some of my earlier posts on this thread about my experiences with bad RAM and mixing kits...

The way I see it, the main point of Xeons is not one thing... it is that they are optimized for enterprise/server market. And for such an application the primary goals are data reliability, system stability, and reaping the benefits of multiple cores/threads. ECC RAM is part of the reliability/stability equation; while overclocking is obviously not. And QPI of course supports stable and efficient expansion of cores/muti-threading.

I hear what you are saying about unlocking Xeons! Wow! I wonder if maybe the complexity of its degradable QPI presents a challenging tuning hurdle, such that it's not worth the testing effort for anything less than a proven market. R&D is expensive; but what do I know... But still, so many peeps these days are into non-gaming, high-perf desktop computing (3D modeling, processing video, folding, etc.). Maybe there's a niche there that might potentially offer some favorable cost/benefit to doing that.

You must be livin right if you were able to get an ASUS engineer on the phone... haha. I'm anxious to hear your report on what ASUS says.


----------



## Migonacci

This maybe a small issue compared to some of the problems other people are having with this mobo. I just recently got this mobo and everything is fine except for some reason the Frnt_fan4 fan connector stopped working. Tried a couple of different fans and nothing. I guess I can check with a voltmeter to see if there is any power coming to the connector. Any ideas?

Specs:
Asus Z9 pe-d8
2x Xeon E5-2670 (ES)
2x corsair h80i
G Skill 64gb 1600 DDr3 Rip jaw
GTX 295
Seasonic 1250w PSU


----------



## anubis1127

Quote:


> Originally Posted by *Migonacci*
> 
> This maybe a small issue compared to some of the problems other people are having with this mobo. I just recently got this mobo and everything is fine except for some reason the Frnt_fan4 fan connector stopped working. Tried a couple of different fans and nothing. I guess I can check with a voltmeter to see if there is any power coming to the connector. Any ideas?
> 
> Specs:
> Asus Z9 pe-d8
> 2x Xeon E5-2670 (ES)
> 2x corsair h80i
> G Skill 64gb 1600 DDr3 Rip jaw
> GTX 295
> Seasonic 1250w PSU


Ugh, I wish that was the only issue I had with this board.


----------



## tomyboy

Anyone experiencing average overall performance in windows 7 or 8?
Like simple tasks, opening windows, application, just simple video playback moving forward and back takes a second?
I just build my second ring 1155 i5 for games and movies and the system is snappy ass hell.. Can not believe the difference in speed, when you open something or scroll a simple video file. I even used a stopwatch, opening some apps and comparing.
In some cases Xeons took twice as long to open something....








Both operating systems are on top SSD with around 500Mb/s read and write speeds tested with ATTO. All the drivers installed, fresh systems and apps on both.
Really have no clue, how the dual xeon system can be all around slower than average 1155 platform with some basic windows tasks. On top of everything I currently use only 4GB of ram on 1155, compared to 64GB with xeons....:








I guess I will be selling this sh..t motherboard an xeons for have the price, if there's even one how will buy it-
In the end, I will be lucky if someone took it for free and saved me from this nightmare called...........


----------



## onmyway

Quick question regarding this board;

Will it boot using Corsair Dominator DDR3 1600? Asking since it's not on the QVL but does cover all the other specs.


----------



## lloyd mcclendon

Probably yes, it should be OK, but don't take my word for it. That's kind of a big risk, but you might get away with it. With ASUS (and probably specifically this board), my experience has been to carefully read their QVL and stick to it to the letter - find the best kit in their listing that hopefully meets your needs.

On the subject of RAM speeds ... I went ahead and picked up the second chip and another RAM kit. My plan was to wait around 6 months for that .. but given how well the first pass went, well I just couldn't wait.







It seemed pointless having the second socket just sit there empty...

So far everything seems fine - except, I cannot get it to post with the memory speed forced at 1600 or above. It worked at 1600 with the clock at 106 with just one CPU. I have the voltages up a bit, I don't think I should need to go further. It sticks at *Code 34 - "CPU post memory initialization"*. Any one work out code 34 yet? I still wanted to call ASUS but forgot, perhaps I will try to discuss this as well as the 600 series on the call.

Also, if you look in the QPI settings in the BIOS, there are items for Link0s, Link0p, and Link1. For some reason the first two are Disabled by default and the third one is Enabled. The manual doesn't explain what the are, any ideas?

It's pulling 985W from the wall under load.... I think I may need a second PSU and proabaly a circuit breaker upgrade before revving the graphics cards..


----------



## lloyd mcclendon

Quote:


> Originally Posted by *tomyboy*
> 
> Anyone experiencing average overall performance in windows 7 or 8?
> Like simple tasks, opening windows, application, just simple video playback moving forward and back takes a second?
> I just build my second ring 1155 i5 for games and movies and the system is snappy ass hell.. Can not believe the difference in speed, when you open something or scroll a simple video file. I even used a stopwatch, opening some apps and comparing.
> In some cases Xeons took twice as long to open something....
> 
> 
> 
> 
> 
> 
> 
> 
> Both operating systems are on top SSD with around 500Mb/s read and write speeds tested with ATTO. All the drivers installed, fresh systems and apps on both.
> Really have no clue, how the dual xeon system can be all around slower than average 1155 platform with some basic windows tasks. On top of everything I currently use only 4GB of ram on 1155, compared to 64GB with xeons....:
> 
> 
> 
> 
> 
> 
> 
> 
> I guess I will be selling this sh..t motherboard an xeons for have the price, if there's even one how will buy it-
> In the end, I will be lucky if someone took it for free and saved me from this nightmare called...........


Well, that certainly doesn't sound right. It's possible you may have some bad circuitry, or have simply done something wrong software wise. To rule out the first part, you might want to try a different OS and see if it performs normally. I wouldn't even want to bother seeing what windows does with this thing.... no offense to the windows users here. But perhaps try out a linux install and see how the basic usage performs. If it still sucks, then perhaps something isn't set in right or you need to get the multi-meter out. If it's normal, you have some .DLL or .SYS problems somewhere.


----------



## onmyway

It did work. However now I am stuck on the issues ASUS is having supporting Gen3 GPU cards. My ASUS GTX670 only works 1 out of 4 reboots as it seems to get stuck in the boot process.
There is a long topic regardig the issue over at ASUS Republic of Gamers;

http://rog.asus.com/forum/showthread.php?27225-ASUS-Z9PE-D8-WS-Issues-detecting-Quadro-K5000-amp-GTX-6XX-series-GPUs-%28Q-Code-62%29/page8

However, no fix as I have been able to find out yet.

Anyone else know about this issues? Some say it might have with the vBIOS BAR size req.


----------



## lloyd mcclendon

EDIT -- nevermind, I'm an idiot. If you want to run memtest on this board, you need to use memtest86+ (PLUS!!) v4.20. The ancient copy of memtest86 (NOT PLUS) v3.3 I had on my usb stick (from my 775 build I think) does not support socket 2011 (to say the least). So it errors out immediately, causing me to think I've got a $5000 brick and tear apart my entire build. After still getting the same with just one socket and DIMM, I had a moment of clarity -- it's just false positives from memtest. Now, memtest86 PLUS v4.20 is running fine, and all appears well. I'll work out the timing/speed/voltage issues soon enough.
--- original post is below









errrrrggghhh ... well ... my prior statement of "being immune to such problems" has came back to bite me

today it failed to come out of suspend .... powered off & booted with the ram speed all the way down as usual, and now it sticks at code 34 even with that ... cleared cmos, booted ok, ran memtest86 and a flood of errors within about 10 seconds









So, now I get to play the fun game of ruling everything out so I RMA the right part. I already removed the 4 new DIMMS and moved 2 of the old ones over to socket 2, and still errors. So most likely all 8 DIMMS are good and I'm suspecting it's the socket 2 half of the board.







I guess I will have to remove CPU2, memtest all 8 DIMMS over in socket 1, and rule those out as good. Then swap CPU1 out with CPU2, and memtest all 8 again to rule out it's the new processor. So if all 8 DIMMS work with both processors in socket 1, then we know it is socket 2 only.


----------



## BlackenedTush

onmyway,

Here's asummary of my similar experience. My issues were a lttle complicated. Hopefully this wordy diatribe will offer up something of help...

My GTX670 is the EVGA SuperClocked 4GB w/ Backplane (NVIDIA driver version 9.18.13.1070), It looks like a reference card. Not sure if brand or model makes much difference with our board since it follows the NVIDIA reference design for the most part, but it is generally touchy with these Keplers. My BIOS is v0503.

*Fixing Posting and GPU Issue*

After putting my beast together I also had problems getting my board to post. Not seeing any 64GB kits out there I had purchased two 32GB Vengeance RAM kits. I assumed that they would play well together. At the time I was clueless as to the cause of the posting issue. But by swapping RAM sticks in positions1 & 2. I was able to get posting to happen; but then my monitors only came to life maybe 2 out of the next 10 attempts and then not at all. My monitors kept complaining of no HDMI signal. I put the GTX670 in various PCIe slots with no better results. EVGA helped me. The problem there turned out to be the fact I had tightened my Dynatron CPU coolers too tight. Evidently the excess tightness was warping the CPU seats on the board just enough to cause PCIe anomalies. So I loosened them up to the point that they weren't pressuring the CPUs at all; but not so much that there was play between the CPUs and seats. I then tugged very gently on the CPUs to equalize pressures and re-tightened the coolers a small amount. My system posted, booted into Windows, and my monitors worked!
After the scramble to get it to post, my GPU had ended up in slot 3, so I moved it to slot 5 where I had planned to put it for better cooling distribution and because it doesn't switch down to 8x based on the presence of other cards in even numbered slots. Further tests confirmed that everything was still working.
Now at this point I was thinking that my problems were only due to CPU seat warping causing issues with the PCIe and RAM channels. I hadn't realized that I had also been dealing with some intermittent problems in the RAM itself.

*Further Problems*

Some days later as I worked to optimize my system and apps, I began to experience corruption in the OS, apps, and files. It wasn't nearly as stable as I'd have liked. But I was so anxious to see how this beast performed by bench-marking and crunching video & 3D. After a few tests Photoshop got corrupted somehow and required a re-install. 3DsMax wasn't working at all and no number of applied patches fixed the issues totally. I was suspecting RAM problems but wasn't certain. Occasionally Window was getting corrupted and I had to revert to earlier system states and/or an older image. Something was just not right. As I continued to try work things on a software settings level, I downloaded some benchmark/diagnostic programs.
Sandra crashed when I tried to view memory and 3DMark11 would run but wouldn't post results up to their website. FutureMark said that the non-posting was due to bad RAM or a bad boot drive. Piriform Speccy indicated just half of my RAM capacity in slots 1 and 2. When I looked in BIOS I got the same capacity readings. So I swapped sticks1 and 2 again. Surprisingly it booted (considering this was the config I had when it wouldn't post originally). BIOS indicated that it had all of its capacity back; but strangely enough Speccy still listed only half capacity on those slots. I tried 3DMark11 again and still no results posted.
So I created a MemTest boot disk and ran it overnight to really give it a workout and to get an overall view of RAM health. In the morning I was stupid and forgot that I had turned off my monitors. So I thought that it had crashed. Of course when I pushed the boot button nothing happened. So I shut it off using the back switch. It booted and I realized my monitors were off. I was able to check system logs to see if MemTest had saved a log, but as I explored Windows crashed. Then it wouldn't even boot. Attempts at reverting to earlier states did nothing and it BSOD'd. Crap! At that point I suspected I had pushed any marginal RAM into failure and that I most likely had corrupted Windows beyond repair.

*Stable Machine*

Before wiping my boot drive and reinstalling the OS/apps, I knew I needed to verify the RAM as being a cause of instability, corruption, and my 3DsMax issues. Using my MemtTest boot disk I was able to test each RAM stick one by one without Windows, and as I was doing so I called Corsair. They told me that there were two potential issues: 1) Mixed 32GB kits, and/or 2) bad RAM. He explained that when mixing kits there are very likely compatibility issues at the advertised speed/frequency/voltage settings. That's because there's a lot of variation between sticks. And stable faster kits are assembled through rigorous compatibility testing/selection. Different kits are never tested/verified to work together at those high levels. He indicated that I should I should have purchased a single 64GB kit. His go forward suggestion was to first finish my MemTests. Any bad RAM sticks would be replaced by Corsair with a new kit. And if no RAM was found to be marginal or bad, I could fix compatibility issues by lowering the common denominator. That would mean upping the voltage and/or reducing speed/frequency. It turns out that I had 2 very bad sticks in one kit and one fairly bad one in the other. I was in luck - they were obligated to send replacements for both of my kits. But they were courteous enough to replace the two 32s with a more pricey single 64GB kit instead. I wiped my boot drive and reinstalled everything. I let BIOS auto-detect voltage and latency, but had to manually up the speed to the rated 1600. That solved all of my problems. Things are now stable. It's now a great machine. Most of it's snappiness is due to launching of apps from the SSD boot drive. The processor clock speed of my e5-2630s is high average. Switching between apps and my browser I noticed that focus takes longer to come to the browser than with a lot of single CPU units - there must be a lot of handoffs in such a complicated system; but heavy duty processing tasks are way faster.

*As for the GTX670 GPU...*

3DsMax works (with viewports set to Direct-X 10). 3DMark11 is primarily driven by the awesome GTX670 and works well. Adobe Apps work well, including GPU acceleration of Premiere's "Mercury Playback Engine" that relies on heavily on technologies like CUDA in the card. I just needed to hack it's list of accepted GPUs to be able to select it in the preferences. And simultaneous multitasking/rendering is acceptably efficient with the 12 cores and 64GB RAM it's got. I'm assuming that After Effects uses the GTX670 as the primary processor when I enable "Use OpenGL Renderer" in "Render Settings". I've noticed that it's a touch faster than without it selected; but also that particular layers in my complicated composites don't show up in the rendered output. So some further research and experimentation with app performance settings might be in order if I think that I need the flexibility.

*My Advice*

Not knowing what you've done or not and what you know or don't know... I don't wish to insult anyone's intelligence. Given that this board seems haunted at times or at least hormonal, this is what I would do taking each subsequent step if posting doesn't happen:

Clear RTC RAM. Make sure jumper ends up in correct position
Check and verify that CPU coolers are minimally tight
Reduce PCIe occupation to only the GPU. Try GPU in odd-numbered slot 5 or 7 (both are 16x, driven by CPU2, and not subject to speed switching complications)
Try GPU in odd-numbered slot 1 or 3 (both 16x/8x, driven by CPU1, and subject to speed switching complications)
Reduce RAM to a single stick in slot DIMM_A1 (the leftmost slot for for CPU1)
Swap alternate RAM sticks into DIMM_A1
Swap GPU into the alternate PCIe slot (5 or 7)
Reduce to most minimal config: only one CPU, one stick of RAM, and one PCIe card (GPU).
Single CPU must be in seat CPU1 (upper right), Single RAM stick must be in slot DIMM_A1 , and GPU must be in odd-numbered PCIe slot 1 or 3 (where it's driven by CPU1).
Try swapping in other RAM sticks into DIMM_A1 and moving GPU to the other odd-numbered slot
If you get it to post at most minimal config, then populate with additional RAM (from one kit). Begin by making a 2x config - add DIMM_B1 and test it, Then make a 4x config by adding both DIMM_C1 & DIMM_D1 and test. If that works, try adding the second CPU.

Good luck!


----------



## Donkey1514

Anyone gotten a Q code of "00"? I bought some 2680 ES chips and installed them and when I try to boot all I get is that code..... I double checked all connections and cleared the CMOS several hundred tmes and nothing works. Any help is appreciated.


----------



## lloyd mcclendon

I've seen that once before but it did not stick. Since that one is first in the sequence, "Not used", and before 01, "system power ON" - I'd point to bad power delivery, a bad connection, a short (extra standoff under there?







), or a bad cpu install -- and ruling all that out, a faulty component. It's not bios settings or anything as it's no where near that point.

----
Had a call with asus tech support today. It could have been a lot worse, the guy knew what he was doing at least. His recommendation was to just put everything back to the defaults and roll with it for a while. Especially ram timings. Though the ram says 11-11-11-31 on the box, the board initally picks up 9-9-9-24 and now it wakes up out of suspend - I suspect the freezing issue issue may be fixed as well (though time will tell). . Any thoughts - what's the deal with mfg recommended ram timings vs what the board wants, I thought you should *always* set what the ram mfg says?


----------



## Donkey1514

Quote:


> Originally Posted by *lloyd mcclendon*
> 
> I've seen that once before but it did not stick. Since that one is first in the sequence, "Not used", and before 01, "system power ON" - I'd point to bad power delivery, a bad connection, a short (extra standoff under there?
> 
> 
> 
> 
> 
> 
> 
> ), or a bad cpu install -- and ruling all that out, a faulty component. It's not bios settings or anything as it's no where near that point.


well it's running on a brand new Seasonic X750. I double and triple checked all power connections....


----------



## lloyd mcclendon

are you sure you have the clear cmos jumper in the normal position

Code:



Code:


----------
|  cpu cpu
|
|pci         _
|          [o_o]  o
|pci
|----------


----------



## Donkey1514

Quote:


> Originally Posted by *lloyd mcclendon*
> 
> are you sure you have the clear cmos jumper in the normal position
> 
> Code:
> 
> 
> 
> Code:
> 
> 
> ----------
> |  cpu cpu
> |
> |pci         _
> |          [o_o]  o
> |pci
> |----------


yea it's covering the middle and left pins


----------



## Trotador

Quote:


> Originally Posted by *Donkey1514*
> 
> Anyone gotten a Q code of "00"? I bought some 2680 ES chips and installed them and when I try to boot all I get is that code..... I double checked all connections and cleared the CMOS several hundred tmes and nothing works. Any help is appreciated.


Are your 2680 ES C0 stepping?
What BIOS are you using?


----------



## lloyd mcclendon

He probably doesn't know what version of bios it is since it has barely even powered on yet.







I'd start to question the power supply .. Are you sure it's a good one, have you tested it with another machine and/or a power supply tester? Are you sure that model is heavy duty enough to run this piece with two processors? I assume it has 2x 8pins and you have both securely plugged in on both ends right







Can you borrow a different bigger PSU and try that...


----------



## tomyboy

Quote:


> Originally Posted by *lloyd mcclendon*
> 
> Well, that certainly doesn't sound right. It's possible you may have some bad circuitry, or have simply done something wrong software wise. To rule out the first part, you might want to try a different OS and see if it performs normally. I wouldn't even want to bother seeing what windows does with this thing.... no offense to the windows users here. But perhaps try out a linux install and see how the basic usage performs. If it still sucks, then perhaps something isn't set in right or you need to get the multi-meter out. If it's normal, you have some .DLL or .SYS problems somewhere.


I did nothing wrong. Same problem with WIndows 7 or 8. Now I tested 3dsmax and it is impossible to work with some moderate scenes, nothing complex. Viewport give me about 1FPS.








Machine is now in shop to figure out what to do. I put inside GTX 690 so the motherboard really wont work...








On thing I did try and it is just unbelievable. We installed ATI graphic card for 50$ on the same OS, just installed new ATI drivers and run the 3ds max with same project......... And guess what ... viewport gave me smooth 30FPS. I could scrub the timeline no problem.
So my question is, am I stupid after so many computers I have assembled or this motherboard doesnt work like it should with any nvidia card due to this PCI Express issues?
Difficult matter to claim RMA, because on a first glance works OK..







Why didn't I go for simple HP Workstation and pay a little more......


----------



## Donkey1514

Quote:


> Originally Posted by *Trotador*
> 
> Are your 2680 ES C0 stepping?
> What BIOS are you using?


how can I find out the stepping?

Here's a pic of one chip



Quote:


> Originally Posted by *lloyd mcclendon*
> 
> He probably doesn't know what version of bios it is since it has barely even powered on yet.
> 
> 
> 
> 
> 
> 
> 
> 
> I'd start to question the power supply .. Are you sure it's a good one, have you tested it with another machine and/or a power supply tester? Are you sure that model is heavy duty enough to run this piece with two processors? I assume it has 2x 8pins and you have both securely plugged in on both ends right
> 
> 
> 
> 
> 
> 
> 
> Can you borrow a different bigger PSU and try that...


I tried a 1000w psu and got the same result.


----------



## anubis1127

Quote:


> Originally Posted by *Donkey1514*
> 
> how can I find out the stepping?
> 
> Here's a pic of one chip


Put it in your x79 board, and fire up CPU-Z. xD

(or you can check here: http://www.cpu-world.com/sspec/QB/QB79.html )


----------



## Trotador

Quote:


> Originally Posted by *Donkey1514*
> 
> how can I find out the stepping?
> Here's a pic of one chip
> 
> 
> 
> I tried a 1000w psu and got the same result.


Yeah, it is a C0, they work on any dual board.

Problem should be elsewhere. I do think it is not power suppy related. is it on a case or just bare nude on some surface?, watch out that there is not any short circuit under the board.. I assume you have checked memory. Did you try with the board VGA? 1 processor at a time?, you know the standard stuff to isolate errors


----------



## balnazzar

Can someone make a summary of pros and cons of this board, and write down a list of potential problems?

I'm purchasing this board, but looking at this thread, i'm now a bit scary.

Can you recommend, after all, the board? I choosen this simply beacause I don't see alternatives on the market.


----------



## Donkey1514

New Z9 fixed the problem!







Thanks for all the help


----------



## ZDngrfld

Quote:


> Originally Posted by *balnazzar*
> 
> Can someone make a summary of pros and cons of this board, and write down a list of potential problems?
> 
> I'm purchasing this board, but looking at this thread, i'm now a bit scary.
> 
> Can you recommend, after all, the board? I choosen this simply beacause I don't see alternatives on the market.


Do you need a 4mhz overclock? Do you need 7 PCI-e x16 slots?
If no to both, get a Supermicro board.
If you need the PCI-e slots, maybe look into a SR-X. Tyan makes some decent boards too.
The Z9 seems to either work or it dosen't. More often than not, at least for me, they don't work until you go through the whole RMA process...


----------



## balnazzar

I simply need decent computational power, two pciex slots and a lot of ecc ram (more than 32 gb). Is a shame that the market offers so few mobos for building a decent workstation. I considered Z9PA-U8, but found only one review at newegg: http://www.newegg.com/Product/Product.aspx?Item=N82E16813131915, and it scares me.
Supermicros 2011 boards (again through newegg reviews), seem to have an high DOA ratio.
I'm considering even an 8-core amd 63xx solution, but again there are few boards on the market. Seems that if you want a decent workstation, you have to spend around 5000$ for a dell/hp machine, with all compromises that come with such a solution


----------



## Goudman

I am evaluating how and if I need to take advantage on the Onboard SSD Cache that comes with the Z9PE-D8 WS motherboard. My thoughts have been to use the following drives to support the onboard Caching: 2 Ea - Samsung 840 Series 256 GB SSD's + 1 Ea - WD 500 GB HDD (10k RPM). My goal is to take advantage of the claim by ASUS that the on board Caching will speed up my system by accessing frequently used data. Please share thoughts on whether the Onboard Caching is worthwhile(?). Thanks

Secondly, I am looking to get advice on where to place my OS and programs SSD drives on the board. My plan is to use similar SSD drive set up for the OS + programs however I was contemplating a RAID setup.

Your suggestions and advice appreciated.


----------



## ZDngrfld

Quote:


> Originally Posted by *Goudman*
> 
> I am evaluating how and if I need to take advantage on the Onboard SSD Cache that comes with the Z9PE-D8 WS motherboard. My thoughts have been to use the following drives to support the onboard Caching: 2 Ea - Samsung 840 Series 256 GB SSD's + 1 Ea - WD 500 GB HDD (10k RPM). My goal is to take advantage of the claim by ASUS that the on board Caching will speed up my system by accessing frequently used data. Please share thoughts on whether the Onboard Caching is worthwhile(?). Thanks
> 
> Secondly, I am looking to get advice on where to place my OS and programs SSD drives on the board. My plan is to use similar SSD drive set up for the OS + programs however I was contemplating a RAID setup.
> 
> Your suggestions and advice appreciated.


If you're going to have two 256GB SSDs, why not just run those in a RAID 0 array? That'll be a lot faster than using a SSD for caching and have the same drive space as a 500GB velociraptor.


----------



## lloyd mcclendon

^ agreed

I would say to avoid use of the Marvell controller. As usual and just as I expected, when that is enabled it breaks the VT-D / IOMMU support since their firmware is *ALWAYS OUT OF SPEC* just as it was on my last ASUS board. So if you need to use VT-D, you have to disable the Marvell controller in the bios. I believe the SSD caching runs through there. As I already eluded to, marvell sucks. period.

SSDs are getting cheap enough and if you can afford this board with 2 xeons, you probably can eventually afford 4 or 8 decent sized SSDs for a RAID0 or RAID10 setup. Forget spinning drives unless you need 4 of the 2 or 3 TB ones right. In any case you should be able to keep your core files on the SSD and your 'data' on the spinning drives, and most likely the caching wouldn't do a whole heck of a lot anyway given the IO usage patterns.

You could also set the same thing up using software raid with --write-behind and --write-mostly and we've already discussed why software raid is *almost always better than hardware raid. flame on

apologies in advance for my tone, I'm about to throw this whole machine out the window. It's as if Murphy himself has moved into the case. (none of these problems are related to the board itself thankfully, it's really nvidia...)


----------



## Morachi

Well, Asus kinda went dead on their replies - so i've put my Quadro K5000 up on ebay. If anyone from Australia wants it, here is one looking for a good home... http://www.ebay.com.au/itm/nVidia-Quadro-K5000-GPU-4GB-/111027636842?_trksid=p5197.m1992&_trkparms=aid%3D111000%26algo%3DREC.CURRENT%26ao%3D1%26asc%3D14%26meid%3D6255168636695141619%26pid%3D100015%26prg%3D1006%26rk%3D1%26sd%3D111027636842%26

I might add I did find that the PSU I had died after 2 months of use, it was an XFX Black Edition, which i've replaced with a Corsair AX1200 series (and it has been great).


----------



## winocm

Anyone know if ES B-Stepping (QA8Z/QA8X) CPUs still work with the latest BIOS? I have some spicy Xeons and I want to know if they'll even POST in this board.

Offtopic:

Otherwise, anyone know if any place still sells Supermicro X9DAi's with BIOS v813?


----------



## amorph777

Has anyone installed Windows 8 on that board? I am running Windows 7 for weeks now, but when I try to install Windows 8 I get the Error "Encountered a problem and need to restart" and the setup restarts. Any idea what the problem could be? I am on Bios 3302 btw.

Thanks


----------



## baishikui

Hey all,

Not sure if you're still looking for additional cases to submit to ASUS/nVIDIA/etc, but I am exhibiting similar issues to the ones logged here.

Build Details:

*CPU*
Xeon E5-2687W ES (QA91) x 2
*Cooling*
Corsair H80i x 2
*Motherboard*
ASUS Z9PE-D8 WS
*BIOS*
0703
*SSD*
Samsung SSD 256GB 840 PRO Series
*RAM*
Corsair Vengeance 32GB 4x8GB DIMM
*GPU*
Galaxy GTX 460 (768MB DDR5)
*PSU*
Corsair AX1200
*Case*
Cooler Master Cosmos II
*OS*
Windoze 7 Ultimate

At first I was working with a single CPU and a 32GB RAM kit. The RAM was detected by the system while in a single CPU configuration. However my H80i was screwed on too tightly it would seem, as loosening it appears to have greatly decreased BSODs, although yesterday I seemed to generate the same ugly nVIDIA driver drops that plagued the board prior to loosening the cooling bracket.

While the CPU was screwed in too tightly, I attempted to boot the system with the following GPUs without success in PCIe slots 1 and 3:

2 x 2GB DDR5 GTX 670 by MSI
1 x 2GB DDR5 GTX 680 by eVGA

Once I had unloosened the screws I attempted the same with the:

1x 4GB DDR5 GTX 670 by eVGA

Despite this chip failing to register with my u3011 monitor, Q-code appears in the same/similar to those appearing when I use a Galaxy GTX 460 in PCIe slot 3, (ultimately 62 and then 15 which persists). Additionally, it appears that the OS is booting because when I added an external speaker to the motherboard I could hear the Windows prompt. I'll note that I haven't installed the onboard speaker, although that would likely help in diagnosing the problem. I'm yet unsure whether or not the problem is with the GPU, the board, or the nVIDIA driver.

===edit===

This morning, (admittedly against the advice of this forum) I installed a second QA91 that appears to now be playing nicely with the CPU I had already installed and the Galaxy GTX 460. However now CPU-Z and Windoze are only seeing 24GB of RAM. I'll attempt reseating/moving them around, then I'll check to see if the BIOS is detecting the sticks a little later.

Perhaps a more important update is that I finally got a 600 series GPU to work with the board:

1x 4GB DDR5 GTX 670 by eVGA

The GPU worked fine out of the DVI-D, or DVI-I output (I can't recall which one) although it reduced the resolution to like 1600xwhatever. This was fixed with a simple nVIDIA driver update to 314.22. The GPU is seated in PCIe slot 3 and is the only GPU installed at the moment, (I will likely add a second card later). Has anyone else had success with this configuration already?

More importantly has anyone had success SLIing Titans with this board yet?




Unfortunately the RAM problem persists. Will likely return the Vengeance 32GB kit and purchase a 64GB kit at a higher clock speed, (so I can hopefully achieve 1600). Any recommendations?


----------



## ZDngrfld

Hmm... I do know these boards don't play well with a lot of memory. I'm using mine in a dedicated folding rig and don't have a high end video card in it, just a Quadro NVS 285. I really don't know anyone who's actually using high end video cards on these boards... I've only had my video card in the first PCI-e slot and it always worked there. I'm running the Samsung 30nm RAM running at 1600MHz in mine. Amazon had the 4GB sticks on sale for $15 a pop a while back and I bought 4 or 8 of them. I can't remember what's in it...


----------



## tomyboy

Quote:


> Originally Posted by *baishikui*
> 
> Hey all,
> 
> More importantly has anyone had success SLIing Titans with this board yet?
> 
> Unfortunately the RAM problem persists. Will likely return the Vengeance 32GB kit and purchase a 64GB kit at a higher clock speed, (so I can hopefully achieve 1600). Any recommendations?


My GTX 960 doesn't work on this board. I guess it is 1-5% chance that the Kepler card wiill work with this board for you....









I went and return Corsair 64GB Kit and went for Kingston 2x16GB kit 2133.
The system is much more responsive. Don't know why, because of the faster ram or this motherboard just have performance drop when you use 8x8GB configuration?

Quote:


> Originally Posted by *amorph777*
> 
> Has anyone installed Windows 8 on that board? I am running Windows 7 for weeks now, but when I try to install Windows 8 I get the Error "Encountered a problem and need to restart" and the setup restarts. Any idea what the problem could be? I am on Bios 3302 btw.
> Thanks


Windows 8 works normaly with this board. No problem with installation.
Could be something wrong with your copy of Win8.


----------



## amorph777

Quote:


> Originally Posted by *tomyboy*
> 
> My GTX 960 doesn't work on this board. I guess it is 1-5% chance that the Kepler card wiill work with this board for you....
> 
> 
> 
> 
> 
> 
> 
> 
> 
> Windows 8 works normaly with this board. No problem with installation.
> Could be something wrong with your copy of Win8.


Well I used my original MSDN-Discs to try to install Windows 8 on that board. And I installed Windows 8 on another Board with that Discs without a problem. So you got Windows 8 running? Can you tell me which BIOS-Version you are using? Do you have 2 CPUs installed?
Thanks!


----------



## BlackenedTush

@baishikui
A lot of peeps in this thread using Engineering Sample Xeons have reported random unexplainable problems like yours. I've posted comments to them. Haven'r heard much on eventual solutions if any... And I've posted (ad nauseum) my failures combining 32GB Corsair Vengeance RAM kits, bad RAM, and my eventual success swapping for one good 64GB kit. And I've also posted my experiences with too tight CPU coolers affecting your very card, and my eventual success there as well. Good luck!


----------



## BlackenedTush

@ZDngrfld
After my typical problems getting this beast off the ground, my board plays great with a 64GB 8x8 kit of Corsair Vengeance 1600 RAM. You may be in for some trouble mixing your single sticks on this board. Best of luck to you... If you want, read any of my numerous past posts on mixing sticks of today's high-speed RAM which haven't been tested & verified together as part of a kit.

I have experienced excellent success with my high-end Kepler GTX670 GPU. It's the _EVGA 4GB, Superclocked / Backplane_ model. I have it installed in slot 5.

Here's the simplified summary of the criteria I used to choose slot 5 for my GPU:
Choose the slots that have the simplest config underneath them. Those would be slots 5, 6, & 7 (supported by CPU2).
Unless doing three or four card SLI, stay away from the slots with the most complicated config and overhead. Those would be slots 1, 2, 3, & 4 (supported by CPU1). Of slots 5, 6, & 7, choose the fastest slots, any odd numbered slot (x16). Stay away from the slower, even numbered slots (x8). And of the slots 5 & 7, choose the highest numbered slot that will work with other cards (to help cooling management flexibility)..

And here's the simple summary of the strategy I would have used if I were to build an SLI config like Maximus:
Start at slot 7. Install each additional card into next available odd-numbered slot.


----------



## BlackenedTush

@tomyboy

I've never heard of a "GTX 960"... you meant 690 right? And I'm not sure where you got that 1-5% success rate figure for Kepler cards on this board, but I'm taking it that it's just a random number picked out of pure frustration... I hear you.

About your increased system snappiness... That's cool to hear! It seems to me that the RAM speed could have something to do with it. It varies by the system. But it also seems very possible that the way this board manages RAM between the 2 CPUs causes some overhead that takes time. I'm assuming you have your new 16GB sticks in slot A1 on CPU1 and E1 on CPU2.
Anyway the complication starts with each CPU having it's own integrated memory controller and the board having memory controllers next to each RAM slot, and then there's the Intel EM64T chip managed by CPU1 to support 64bit RAM addressibility, and the QPI bus to handle the necessary communications back and forth between the CPUs. Of course that's a very simplistic view, but the point is that it's pretty complicated. I'd bet that spreading the same capacity over a larger number of RAM sticks multiplies significantly the number of these transactions that must must happen per clock cycle.

Cheers


----------



## tomyboy

Quote:


> Originally Posted by *amorph777*
> 
> Well I used my original MSDN-Discs to try to install Windows 8 on that board. And I installed Windows 8 on another Board with that Discs without a problem. So you got Windows 8 running? Can you tell me which BIOS-Version you are using? Do you have 2 CPUs installed?
> Thanks!


Latest bios, nothing special. Win 8 instalation works OK. I am using 2 CPU-s.

Quote:


> Originally Posted by *BlackenedTush*
> 
> @tomyboy
> 
> I've never heard of a "GTX 960"... you meant 690 right? And I'm not sure where you got that 1-5% success rate figure for Kepler cards on this board, but I'm taking it that it's just a random number picked out of pure frustration... I hear you.


Of course 690...







......pure frustration jap....








Quote:


> Originally Posted by *BlackenedTush*
> 
> @tomyboy
> About your increased system snappiness... That's cool to hear! It seems to me that the RAM speed could have something to do with it. It varies by the system. But it also seems very possible that the way this board manages RAM between the 2 CPUs causes some overhead that takes time. I'm assuming you have your new 16GB sticks in slot A1 on CPU1 and E1 on CPU2.
> Anyway the complication starts with each CPU having it's own integrated memory controller and the board having memory controllers next to each RAM slot, and then there's the Intel EM64T chip managed by CPU1 to support 64bit RAM addressibility, and the QPI bus to handle the necessary communications back and forth between the CPUs. Of course that's a very simplistic view, but the point is that it's pretty complicated. I'd bet that spreading the same capacity over a larger number of RAM sticks multiplies significantly the number of these transactions that must must happen per clock cycle.
> Cheers


I am using 4x4GB kit on CPU1 and another 4x4GB KIT on CPU2, no problem there

What would you say about this simple test?
After effect with lots of plugins:
(2011 Xeon) first start 45 sec, second start 15 sec
(1155 i5) first start 25 sec, second start 8 sec
Using same OS, same AA plugins, same SSD boot drive. All the application take approximately 2x times longer to open on xeon platform. That is now the other thing that is frustrating me.









Currently I don't need GPU precision or power so I just sold my old quadro and bought gaming ATI Card 7950, just to have a card that would work with this motherboard, with more Video Ram for two High Res monitors.
Thanks for tip the which slots to use.
I will try my brothers new 660Ti, which he allso bought today,
Really queries if this card will work? If it will, then we are going to make the exchange, same price, gaming performance even better on ATI, but having CUDA in adobe would not hurt...


----------



## BlackenedTush

For most applications on most systems, the second startup will usually be faster due to caching, etc.. Most are designed to do that, and it looks like that's one thing you are seeing.
But you also see the slower single CPU config beating the faster Dual Xeon config... I was just thinking that here are other things that might be different between your two systems, like virus checkers, and different processes loading, different system settings etc. etc., but still...

When I was researching this board, I heard a lot of talk about boot and load times being longer than normal. And the reasons people often gave were that this board is really a server board. And as we know, servers are not designed for gaming... They're designed for stable and reliable, heavy-duty WORK; not quickness. Servers in general are a more complex mixture of hardware to manage... And with multiple CPUs, they're even more so. Servers do those extra initialization checks for stability, etc., These extra processes take extra time. The external QPI handoffs between the two CPUs during initialization of an application wouldn't happen the same way within the single CPU of the i5 system. It takes longer as the two CPUs work through QPI to split up and balance the load. "Too many cooks cooking the same soup"...







What I don't like is that the general OS tasks are a little hesitant sometimes. Like if I try to grab the top frame of a browser to move it, a lot of times it won't even respond for 3 or 4 mouse clicks. I just look at it like the CPUs are just fighting over trying to please me.









YEP, slower boot and slower application startup on the dual Xeon system.... that IZZZ a little frustrating... unless you've built the system as a media and 3D workstation and that's your focus. If so, you accept that tradeoff on slower loading as you see your big bad dual Xeon system grinding, chewing, and rendering a complex AE file that uses lots of media, lots of layers, blending modes, effects, and transcoding. I'll bet it does circles around the i5 in that kind of a race.

Yeah you are right about having CUDA for Adobe. Hey, if you really wanna get rid of your NVIDIA cards, I'll take em  Adobe apps exploit NVIDIA technology (that includes OpenGL and others). And they do it more and better with each new Creative Suite release. Plus they are improving their multi-threading to split up processing tasks more efficiently between more CPU cores. AE can really use CUDA cores in many more ways with their new redesigned CS6 engine... even if it's supplied by multiple NVIDIA GPUs (see http://blogs.adobe.com/aftereffects/2012/05/gpu-cuda-opengl-features-in-after-effects-cs6.html and http://www.video2brain.com/en/lessons/system-requirements-for-gpu-acceleration-opengl-cuda). It can be pretty cheap to throw a lot of GeForce cards at it. And do as well or better than Quadro, and give Maximus some competition. Just make sure to have the same version of CUDA and similar RAM capacity between cards (AE will set GPU RAM use across the group of GPUs to the same capacity as the card with the lowest amount of memory).

And Autodesk apps use NVIDIA's bag of technologies too... for 3DsMax, Maya, Mudbox, etc.. (stuff like like Direct3D, Direct X, and CUDA for its iRay framework). They claim (even more so than Adobe) that there's some special and necessary magic within Quadro... so adding Tesla cards to get Maximus is the ultimate ticket for speed with quality... Maybe, I don't know.; but thats more than 10x the cost of my GTX670 and it works pretty well. But if you do any of the above, I'm wondering how much ATI cards will do for you...


----------



## BlackenedTush

Is ASUS going to RMA you another mobo? I mean they say nothing in the manual as a warning about how sensitive the CPU seats are to tight cooler connections.

That's interesting what they said to you about the switches... They are right.. But I wonder if they are changing part vendors in an attempt to make the PCIe a bit less problematic...I say that because the mobo schematic in the appendix of the manual does illustrate the switch on slot 3, but it labels it as "1480 (multiplexer) switch" from ASMedia, and not as a switch from PLX. Hmmmm...

In fact, if you do look at the schematic, you'll see that there are two of those switches. And they affect _all four_ of the PCIe slots connected to CPU1. You can see that slots 1 and 3 are set up to get two x8 links each ( x16 speed total). One x8 link comes direct from the CPU and one x8 link comes from the switch. But since the switch has only one x8 link from the CPU, it can either give it to slot 1 or to slot 2; but not to both. Which means... if a card goes into slot 2, the switch has to take back it's x8 link from 1 and give it to 2 instead. The same scenario occurs with slots 3 and 4.

In my thinking, avoiding complication whereever possible when setting up this complicated board is the smarter way to go. That's one big reason why in my criteria for selecting the best spot for the GPU in a 1-GPU configuration, I wouldn't choose any slot from CPU1.
And an additional reason pops up if one looks closer at the schematic - CPU1 is also directly linked to every other subsystem on the board; while CPU2 is merely linked to three very simple un-switched PCIe slots.

So I'd pick any slot managed by CPU2 (5, 6, or 7). And although it doesn't really make much practical difference at all (at least with a card the magnitude of a GTX680 or slower) the odd numbered slots are the faster x16 link. So in my mind slots 5 and 7 are the prime slots for a single GPU config.

Now If you are venturing into SLI... like a Maximus configuration of NVIDIA Quadro & Tesla cards, the manual talks about it in chapter 7. But it only describes doing that with x16 link slots. So again, given that all the odd numbered slots are x16, and the even numbered slots x8, I'd start with slots 7 and 5. And if I was doing three-way SLI, I'd have of course have to add slot 3 out of necessity. And for four-way, I'd have to add slot 1 into thje mix as well. But I'd make sure if I was using slot 3 or 1, that 4 or 2 respectively are kept empty.


----------



## tomyboy

Thanks...







Quote:


> Originally Posted by *BlackenedTush*
> 
> "Too many cooks cooking the same soup"...
> 
> 
> 
> 
> 
> 
> 
> What I don't like is that the general OS tasks are a little hesitant sometimes.


Well said.

Using Nvidia CUDA with MercuryPlayback engine in Premier Pro is OK, but this is only for faster playbeck.
Would expect also to have rendering advantages if someone buys, let say 2-3k$ card configuration. After effect is even worse, only new Ray Trace feature is using Nvidia Cuda Graphics card, no other mayer benefits, like they advertise.
Adobe is advertising how faster applications are with xeons setup?
Weel in practis this is far from the truth, I would say bull...sh....t...








Premiere Pro for example is utilizeing only 15% of the two xeons power when rendering. Slower than with singl normal cpu setup, what is up with that?
This is not only on my sistem, but reed on some forum about the user who is using HP 5k$ workstation, who has the same same problem..
Hope the new version of Adobe Suit will address this problem, especially when rendering final project, to utilize100% of CPU power.


----------



## darkmatter111

Catching up with this thread, loads of great info and experience. I have posted an intro here as I am soon in the next few months going to start a Z9PE-D8 build


----------



## TheBlademaster01

If there is anyone that plans on selling this board please PM me


----------



## TheBlackHole

Hi to everybody!
I encountered several problems with this mainboard and I hope someone here on the forum is able to help me. It will take some time to read all that I wrote but I think if you will be patient, you will agree that is definitely worth it!

I highlighted every part that is important, and my questions are preceded by "Q."

First of all my configuration:

- 2 x Intel Xeon 2687W
- 4 x 8GB Samsung M393B1K70DH0-CK0 modules, ECC / Registered
- 1 x Seasonic Platinum 1000W PSU
- 1 x Samsung 830 SSD
- 1 x NVidia 7800 GTX
- Windows 7 Ultimate 64 bit OS

Please note that *I kept the installed hardware @ the bare minimum , just to test and solve the issues I'm experiencing.*

I used another graphics card from ATI too, cheap but way more recent than the old 7800 GTX.

Also you must *keep in mind that I made a "raw" Windows installation , without any specified or updated driver* for the hardware installed and I did not installed any of the software/drivers on the CDs that comes with the mainboard.

I will try to explain the problems encountered by dividing them into phases, making a chronicle of my experience with this mainboard, both indicating solutions to commonly encountered problems (when I managed to find them!) and asking you tips and explanations for the problems that still are there.

*1)* Initially I could not get a POST no matter what, the mainboard showed on Q-CODE display virtually the entire range of possible error codes, like a causal extraction of numbers.

With both 2 and just 1 CPU installed, and with all 4 or just 1 RAM modules.

I soon realized that the only possible explanation for such behavior was some kind of problem with the memory.

After dozens of attempts, resets, clearing CMOS, etc.. the board still refused to boot.

Then something very strange happened: I tried with some cheap RAM - same result - BUT after that, when I put back the Samsung modules, the mainboard has finally begun to "collaborate" and I had no problems getting a POST and boot.

As if, once "tasted" the worst, she understood that was best for her to be "satisfied" with the ECC / Registered PC12800 (1.600Mhz) modules from Samsung, @ 100 Euro x piece ..........

*2)* After some boots I noticed that the BIOS indicated that portion of the installed memory was not used, or rather seemed "invisible" to the system: at first only a "half" module (the BIOS gave about 28GB available instead of 32GB, so half a module of 8 GB was missed), then after other attempts and resets, even disappeared the entire RAM bank, showing only 24GB (!).

*An option on the menu "Advanced / Chipset / Memory" showed sometimes the message (bank) "Present: 0 MB", sometimes the equally ominous "Present: Disabled".*

Here, too, after dozens of attempts and after reversing the two CPU from socket 1 to 2 and vice versa, I managed to "convince" the mainboard to accept the harsh reality of the existence of 32GB of RAM on the DIMM .......

PLEASE NOTE that I have examined very carefully all the pins of both sockets and all of them are absolutely PERFECTS.

I read on variuos forum that other users have had this same issue with this mainboard and no one was able to find an explanation and it seems that they are not even able to overcome the problem: in this regard I have some luck, at least.....

*Q. Someone can enlighten me on this issue?*

*3)* In the process of course *I tried to update the BIOS from the original 3206 to the newest 3302.....not only did not help to solve the problem of RAM "disappearance", but it has added another*: after each system restart, the mainboard remained "alive" only for a few seconds and then suddenly turn off and restart, repeating the whole loop endlessly.

*The only way to stop this behavior was - @ every single system restart"!!! - the CLEAR CMOS option.*

*Alternatively, I found out that setting jumper ME Recovery on the "enable" - AND KEEPING it as such - allowed the mainboard to systematically avoid the "deadly loop"* but it is clear that this could not be a solution ( I do not think that the ME Recovery jumper must be always on) and the entire boot seemed slower than normal.

*Q. By the way, what is exactly the ME Recovery? Enabling it what exactly does?*

In the end, *I found the definitive solution for pure casualness*: after restoring the original BIOS 3206 and above all *after disassembling the entire system and reversing the CPU from the socket 1 to 2, everything turned back to "normal" in the sense that the mainborad stopped the strange loop on every reboot.*

Note that this issue,too, was already reported by some users, as I found out in other forum.

*4)* *The BIOS shows that the processors speed is 1200 Mhz* (instead of 3100) and also the voltage is much lower than normal (obvious consequence).

*If I do a CLEAR CMOS, the mainboard sets speed again to 3100 Mhz but this ONLY LAST until the next reboot of the system, then lowers again the frequency to 1200 MHz.*

All this does not disturb the performance because obviously Windows autonomously manages the CPUs clock frequency and voltages depending on the workload.

It seems to be a bizarre setting of this mainboard that seems to try to save energy at bootup.

Or is not? It can be a sign that something's wrong? I personally do not think that's the case because this "issue" has already been reported by many users (again!) on other forums and it seems to be a peculiarity of the board or - most likely - of the newer BIOSes.

*Q. Can someone tell me more about it?*

*5)* Now, the problems that remained unresolved.

Various monitoring software in Windows environment (such as CPU-Z, HWINFO64, AIDA64, etc.) show that both CPUs have a frequency of 1200 Mhz (which should be normal in the absence of workloads - *Q. right?*) BUT *all the various cores - almost at random - experience a sudden increase in frequency with "Turbo"activation, but not caused by any software running or any element that can stress the CPUs , along with an equally sudden increase in voltage.*

This phenomenon is repeated continuously without stopping.

Sudden changes of frequency and voltage (about 1.1 V), then return to 1200 MHz and minimum voltage (about 0.8 V).

Initially it did not seem to happen, I noticed this thing after updating the BIOS to version 3302: I've now restored the 3206 but the phenomenon persists.

*Q. Now I ask: is this normal behavior or is a symptom that something's wrong?*

*6)* Again RAM issue, and once again a strange one.

Although now the BIOS always recognizes the 32GB installed, in Windows there is something strange: *some software (HWINFO64, AIDA64) shows 32 GB accurately (32768 Mb) while only CPU-Z show values slightly lower - but not correct (32718 or 32720 Mb, depending on how he feels ...).*

Of course I thoroughly tested each memory module with various software and I did also stress test for over 24h with MemTest86 +,with absolutely NO ERRORS detected.

What's more, one of the CPUs recognizes and correctly handles all modules (whether I entrust her all 4 or just 1), and behaves perfectly on both socket 1 and 2.

While the problem seems to affect only the other CPU, that show it in both sockets: does not hold all 4 modules (i.e. if I install this CPU alone with all 32 GB entrusted to her, at least 1 module of 8 GB "vanishes"...) and also whether I install both CPUs the problem I described above (several megabytes "disappeared") happens, but ONLY if I installed all 32 GB (obviously divided between the 2 sockets).

It seems this CPU has some problem on the memory controller (at least this is my inference) ..... *Q. this is the only explanation or there may be something else?*

Curiously enough, if the "faulty" CPU is, for example, on the socket 2 and I insert the RAM modules on the DIMM "G1" & "H1" rather than the "E1" & "F1" (as described by the manual) CPU-Z shows the correct values (32768 Mb) but, as was to be expected, there is a decline in memory performance.

Another curious thing is that I tried to install both CPUs and put all 4 RAM modules only on socket 1 (where is the "healthy" CPU), leaving socket 2 - where sit the "faulty" CPU - free of RAM : the system recognizes all 32 GB correctly BUT while HWINFO64 & Aida 64 reports them running in Quad-Channel mode (!!!), once again CPU-Z only disagrees and can not indicate the "channel mode" in which they operate, furthermore showing that the RAM run at 500 Mhz instead of the normal 800 (as if they were DDR 1060 instead of DDR 1600).

*Q. By the way, it is correct that the RAM setted on pair on both sockets as the manual reccomend (16 GB on socket 1, DIMM A1 & B1 and 16 GB on socket 2, DIMM E1 & F1) run in Dual-Channel mode (as happens in my mobo)?*

*Q. There is a way to run them (4 modules x 8 GB) in Quad-Channel on this mobo or I need 8 modules and fill all the RAM DIMM on both sockets?*

I did some tests and benchmarks and there not seems to be big difference between the "correct" RAM configuration (from the manual: 16 GB on socket 1, DIMM A1 & B1 and 16 GB on socket 2, DIMM E1 & F1) and the wild one tried by me ... although there are some inconsistencies: benchmarks from AIDA64 not seems to feel any differences, while GeekBench do not like at all the 4 modules on a single socket and heavily penalizes the score.

Finally, PassMark tests are the most ambiguous because PCMark (evaluation of CPUs) approves and returns a very high score (which I believe is not reliable) while MemMark (of the same package, which assesses the memories) returns very poor values for the RAM performance.

*Q. Therefore, I believe that is not a good solution to put all 4 modules on sockets 1 and probably it may cause a performance decline: what do you think?
*
Last question: when I install one CPU alone, even with just 1 RAM module, AIDA64 benchmarks for memory test show decent values - for ex., let's say 15.000 on "Memory Read test" - but when I install both CPUs, even with all 32GB of RAM, I get a sensible drop in performance - like 11.800 in the very same test.

Also "Memory Write & Copy test" values drops considerably.

The latency rises from, let's say, 69ms to 80.

And this happens no matter what (switching CPU from socket 1 to 2 and viceversa, switching memory modules, less modules, more modules, BIOS setting,etc.).

*Q. Is it normal? I mean, with 2 CPU is somewhat "physiological" a decline in memory performance? Or ,on the contrary, it is a sign of something wrong?
*

*7)*Last "issue". *As soon as I see the Windows logo on my monitor, the Q-CODE display of the board show value 22*, and stays so forever.

I have not tested the system 24h/7, only some benchmark & simple software, so I can't be 100% sure that it is absolutely stable.

Altough I feel optimistic about that.

*I did some search on internet and found out that no one can say clearly what this code 22* is, which is the cause and *especially if it is a real problem or not*: I've read about people who had this same report but whose system seemed stable at 100%.

*Q. Somenone here has an exhaustive answer? And, by the way, what should be the "correct" code (ie that everything is ok in the system) that the Q-CODE display should show after boot?*

Last notes: trying to solve all the issues described above I did several tests, for example using a video card, then the other one, moving them in each slot to see if there were benefits.

I've tried changing various parameters in the BIOS: first all on "Auto", then I tried to manually setting the values of the RAM timings, speed and voltage.

Disabled PCIE slots not being used, disabled the onboard video. And vice versa.

*Q. Is there anything else I could try?*

By the way, I also noticed that some of the CPU settings such as QPI Link 0s and 0c are set by the BIOS to "disabled", while only QPI Link 1 is "enabled": *Q. is this normal? It's useful to enable the other two settings? What are they? it is better to disable all 3?*

I made a few attempts but do not seem to have any effect neither positive nor negative on the system ....

If you have read so far, thanks!!! I hope someone can help me. *Thanks in advance for every reply.*


----------



## ZDngrfld

The one thing I've noticed that can cause issues like this is if you have your heatsinks tightened down too tight. I haven't used ECC memory with this board so I can't help there. The QCode that will be displayed once it has finished POSTing is FF.


----------



## TheBlackHole

Thank you very much for your reply!

Yes, I know about some issues being related to the heatsinks tightened down too tight and to tell the truth I think that loosened them helped to fix the problem of memory banks that were not recognized.

But I do not think I can loosen them more than that









Also, the problems related to the memories appear to be linked to only one of the CPU.

Moreover, even installing only the "healthy" CPU, the Q-Code 22 remains on display. (even with just 1 memory bank)

I did not know that "FF" is the final code and, by the way, I saw all genres of code on the display but never the FF....


----------



## ZDngrfld

Quote:


> Originally Posted by *TheBlackHole*
> 
> Thank you very much for your reply!
> 
> Yes, I know about some issues being related to the heatsinks tightened down too tight and to tell the truth I think that loosened them helped to fix the problem of memory banks that were not recognized.
> 
> But I do not think I can loosen them more than that
> 
> 
> 
> 
> 
> 
> 
> 
> 
> Also, the problems related to the memories appear to be linked to only one of the CPU.
> 
> Moreover, even installing only the "healthy" CPU, the Q-Code 22 remains on display. (even with just 1 memory bank)
> 
> I did not know that "FF" is the final code and, by the way, I saw all genres of code on the display but never the FF....


Yeah, here's my Z9 folding away showing FF


Spoiler: Warning: Spoiler!






. Have you tried both CPUs in the first socket by themselves and have just one stick of memory? I wasn't sure if you tried that when I saw you were trying all your 8GB sticks with one CPU. This motherboard is really picky with what memory it will work with. Are the sticks you are using on their "QVL"? If you want quad channel, you'll have to run 8 sticks of memory.


----------



## TheBlackHole

1) Yes, I tried both CPU on socket 1 and yes even with just 1 stick of memory. All four sticks on the same socket was just an experiment









2) No







, these memory are not there. BUT I found several reviews on specilized & reliable web-sites in which they used exactly this RAM and the mainboard gave no problem at all.
Instead, one of them even showed the great overclocking potential of these ram in conjunction with the Asus Z9PE-D8 WS









Can you tell me something about the sudden CPUs oscillation of frequency and voltage in Windows? (NO workload, NO software running: read my first post here for further explanations) Is it normal? Because in my firsts boots I did not noticed this phenomenon and now I'm not sure if has always been there









Thanks


----------



## ZDngrfld

Quote:


> Originally Posted by *TheBlackHole*
> 
> 1) Yes, I tried both CPU on socket 1 and yes even with just 1 stick of memory. All four sticks on the same socket was just an experiment
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 2) No
> 
> 
> 
> 
> 
> 
> 
> , these memory are not there. BUT I found several reviews on specilized & reliable web-sites in which they used exactly this RAM and the mainboard gave no problem at all.
> Instead, one of them even showed the great overclocking potential of these ram in conjunction with the Asus Z9PE-D8 WS
> 
> 
> 
> 
> 
> 
> 
> 
> 
> Can you tell me something about the sudden CPUs oscillation of frequency and voltage in Windows? (NO workload, NO software running: read my first post here for further explanations) Is it normal? Because in my firsts boots I did not noticed this phenomenon and now I'm not sure if has always been there
> 
> 
> 
> 
> 
> 
> 
> 
> 
> Thanks


That would be Speedstep. When you're in Windows does the frequency ever hit 3.4GHz when they're under a full load?


----------



## TheBlackHole

Quote:


> Originally Posted by *ZDngrfld*
> 
> That would be Speedstep. When you're in Windows does the frequency ever hit 3.4GHz when they're under a full load?


I know SpeedStep and as I described in my first post, it seems that the Turbo turns on suddenly and randomly: SpeedStep should activate it only when is needed (heavy workload, CPU burning, benchmarks,etc.) but in my case there are NO software and NO benchmarks running.

Just after Windows load finish, if I launch one of the many monitoring tools that I have installed (like CPU-Z or HWINFO64) these show sudden changes in frequency in both CPU accompanied by corresponding changes in voltage (obvious consequence).

As I wrote, at first I did not think this to happen: can you please verify if in your machine (just after Windows load) the same phenomenon happen? (without running software that require Turbo activation!) If not, the frequency of yours CPUs stay still on 1200 Mhz?

The best utility that you can use for check this, is HWINFO64: so please if you don't have it, download it (it's absolutely free).

This way you can assure me on the matter.....or maybe not!


----------



## ZDngrfld

Quote:


> Originally Posted by *TheBlackHole*
> 
> I know SpeedStep and as I described in my first post, it seems that the Turbo turns on suddenly and randomly: SpeedStep should activate it only when is needed (heavy workload, CPU burning, benchmarks,etc.) but in my case there are NO software and NO benchmarks running.
> 
> Just after Windows load finish, if I launch one of the many monitoring tools that I have installed (like CPU-Z or HWINFO64) these show sudden changes in frequency in both CPU accompanied by corresponding changes in voltage (obvious consequence).
> 
> As I wrote, at first I did not think this to happen: can you please verify if in your machine (just after Windows load) the same phenomenon happen? (without running software that require Turbo activation!) If not, the frequency of yours CPUs stay still on 1200 Mhz?
> 
> The best utility that you can use for check this, is HWINFO64: so please if you don't have it, download it (it's absolutely free).
> 
> This way you can assure me on the matter.....or maybe not!


I don't run Windows on that box. It's running Debian. Mine doesn't throttle to 1.2Ghz. It steps down to 2Ghz at idle, which is what the default frequency is for a E5-2650. Can you check the multiplier in the BIOS? I don't know why that would have changed, but it almost sounds like it has.


----------



## TheBlackHole

Ah ok, we have different CPUs.

The BIOS show 1.200 Mhz: I have no option about the multiplier (is setted automatically by the mainboard I think)----> In my first post I wrote also about this strange behavior, the BIOS @ first set the frequency to the "normal" value (3.100 Mhz) BUT as soon as I reboot the system the BIOS change the frequency to 1.200 Mhz and remain the same unless I do a CLEAR CMOS.

Alternatively I need to disassemble the ring and remove the CPUs from sockets: this way, after reassembling the system, the BIOS set again the frequency to 3.100 Mhz (only until the first reboot...).

I read all this happen to other users too, so I thought it was a peculiarity of the new BIOSes....what about you? which BIOS version you use?


----------



## ZDngrfld

Quote:


> Originally Posted by *TheBlackHole*
> 
> Ah ok, we have different CPUs.
> 
> The BIOS show 1.200 Mhz: I have no option about the multiplier (is setted automatically by the mainboard I think)----> In my first post I wrote also about this strange behavior, the BIOS @ first set the frequency to the "normal" value (3.100 Mhz) BUT as soon as I reboot the system the BIOS change the frequency to 1.200 Mhz and remain the same unless I do a CLEAR CMOS.
> 
> Alternatively I need to disassemble the ring and remove the CPUs from sockets: this way, after reassembling the system, the BIOS set again the frequency to 3.100 Mhz (only until the first reboot...).
> 
> I read all this happen to other users too, so I thought it was a peculiarity of the new BIOSes....what about you? which BIOS version you use?


I would have to see what it's running. I'm not at home to check, though.
I never had an issue with it reporting an incorrect frequency. I had issues with my memory reporting missing modules, like you have. I RMAed the board after it failed to boot for no apparent reason a few hours after I had it working. No QCodes or anything, just quit booting all together. I got the replacement and never messed with it again. It's running whatever version of the BIOS they sent with it. It booted and has kept running ever since. Are you certain your CPUs aren't the issue? If so, RMA the board.

I'm staying far far away from Asus and their LGA2011 workstation/server products. My latest 2P LGA2011 setup is using a Supermicro motherboard and I haven't had one issue with it at all. It also has a IP KVM built into it. You don't have to pay ~$50 for a little module like you do with Asus.


----------



## TheBlackHole

Quote:


> Originally Posted by *ZDngrfld*
> 
> I'm staying far far away from Asus and their LGA2011 workstation/server products. My latest 2P LGA2011 setup is using a Supermicro motherboard and I haven't had one issue with it at all. It also has a IP KVM built into it. You don't have to pay ~$50 for a little module like you do with Asus.


Yes, you are absolutely right. I had a poor experience with Asus and their customer service.....

I think too that the Supermicro motherboard is way less problematic and more reliable but is not so easy to buy here in Europe









By the way, my ram sticks are the FIRST choiche recommend by Supermicro on their mobo for Xeons E5........

RMA the board will be a real pain, I got lot of experience precisely with Asus.....I would be sure that the board is somehow defective but I got the feeling that the BIOS is the main problem here.

The older versions (like 0503 for ex.) gave far less problems: but this board has a newer version (.CAP instead of the older . ROM) and I cannot downgrade the BIOS!!! I can upgrade only.

An idea is to buy an EEPROM chip with an older BIOS version and install it: I read that some users have solved their issues with this solution


----------



## ZDngrfld

Quote:


> Originally Posted by *TheBlackHole*
> 
> Yes, you are absolutely right. I had a poor experience with Asus and their customer service.....
> 
> I think too that the Supermicro motherboard is way less problematic and more reliable but is not so easy to buy here in Europe
> 
> 
> 
> 
> 
> 
> 
> 
> 
> By the way, my ram sticks are the FIRST choiche recommend by Supermicro on their mobo for Xeons E5........
> 
> RMA the board will be a real pain, I got lot of experience precisely with Asus.....I would be sure that the board is somehow defective but I got the feeling that the BIOS is the main problem here.
> 
> The older versions (like 0503 for ex.) gave far less problems: but this board has a newer version (.CAP instead of the older . ROM) and I cannot downgrade the BIOS!!! I can upgrade only.
> 
> An idea is to buy an EEPROM chip with an older BIOS version and install it: I read that some users have solved their issues with this solution


Yeah, I had a hard time getting the board replaced. They originally sent me emails and wanted to start the RMA that way. They couldn't even find my motherboard by the serial number that was on the motherboard! I even went as far as to send them pictures of the serial numbers, both on the box and on the board itself. I had to call in and deal with someone completely different to get it done. Even after that they sent me the wrong forms to fill out... It was a joke dealing with them and I don't plan on buying anything enterprise related from them in the future. Getting a new BIOS chip is an option. Have you found where you can get one for a decent price? I'd give it a shot if you did. I know a couple people in the states that have purchased them for around $10.


----------



## TheBlackHole

Quote:


> Originally Posted by *ZDngrfld*
> 
> Getting a new BIOS chip is an option. Have you found where you can get one for a decent price? I'd give it a shot if you did. I know a couple people in the states that have purchased them for around $10.


Yes, thank you. I found a shop in Germany that sell the chip for 10 Euro (about $13), shipment included.

Thanks again for all your advices and for your time.

If there will be (good) news'll keep you informed!


----------



## AndyE

Quote:


> Originally Posted by *TheBlackHole*
> 
> Hi to everybody!
> I encountered several problems with this mainboard and I hope someone here on the forum is able to help me. It will take some time to read all that I wrote but I think if you will be patient, you will agree that is definitely worth it!


I do have 2 ASUS systems Z9PE-D16 and Z9PE-D8 with 2x E5-2660 and 2xE5-2687W which i regularily change in between. Main memory is 256GB and 128 GB respectively, ECC 1600MHz.

To your questions:
1) The Q-Codes are listed in the manual and provide at least a solution to know until when the POST run ok. Good that the system is now starting
A good summary for initial boot issues:
http://www.tomshardware.co.uk/forum/261145-31-perform-steps-posting-post-boot-video-problems

2) If memory is "missing" then the first thing I would do is to reset the CMOS memory. Potentially, in trying to get the system starting, you changed essential settings and impacted this part of POST

3) ME is the Intel Management Engine which is part of Intels vPro offering. Basically it is a small computer on the motherboard supervising the operation of the system, even if the main CPUs are turned off - i.e. you can remotely start the Asus MB via this mechanism

4) With default CMOS settings, the MB shows the tarbet base frequency, which is in your case 3100MHz. If perf mode and similar things are changed, it could be different (haven't personally tried it)

5) This is normaly behavior. Even if the workload of AIDA, HWInfo etc is not big, you are executing software on your system. There is no hard affinity between the executing thread in an application abnd the physical core in the socket - shows up as a kind of random pattern. Benchmark software potentiall wakes up the CPU to get the timing operation right. I would *only* be concerned, if the system with good benchmark software under load would not go to either 3100MHz or better 3400 MHz (depending on your cooler). 3400 MHZ is the max frequency for all cores in Turbo mode. For 1-2 core under load the speed can go up to 3800 MHz. Good benchmark software for such a system is Intel's Linpack, be more critical of the results of the usual known benchmark apps - they are not optimized for NUMA boards like yours.

6)
Memory amount: This is normal
Memory is used for all kind of things like I/O devices are mapped into physical memory space. Some utilities see one POV, some have other approaches. 48 MB difference look like a PCI express card is mapped into memory space.

4 x 8 GB memory is a suboptimal configuration for this system. One of the reasons for a dual socket MB is a the increased bandwidth you might get. In this configuration the 16 cores of the 2xE5-2687W will starve for memory bandwidth to achive their regular performance. Basically, you run the 8 core socket with the memory bandwidth of a 4 core i7-3770. Either switch to 8x4 GB or 8x8GB. Systems like this are NOT designed to achieve low latency /high speed for individual threads, for these kind of apps an OC system with fewer cores is the better solution. If you need THROUGHPUT, then this system will shine and and leave all single socket systems in the dust. As an example: It is possible to read a 100GB file within 5 seconds into main memory (this is actually what I am doing with mine). Try this with an OC i7-3770K or i7-3930K, it wouldn't be possible.

To identify possible errors, I'd start with one populated socket, and go with 1 RAM DIMM and start there. Run the system under load for a few hours (linpack i.e.) and check the stability in its minimal config. Move from there by adding more memory Dimms to one socket. If all goes well, THEN add the second. If problems arise, change back to a single socket config with the "faulty" CPU and re-check stability. If stability is there, your setup might have been screwed for dual socket ops (or one CPU is indeed defect. Clear the CMOS and re-check with 2 sockets.

As said, many classic benchmark utilities are "confused" with NUMA systems or if the performance goes in regions beyond the design of the software. Some of them get actual slower (as data has to be moved via the QPI interconnect and there are WAY too many cores to choose from. Software which does not explicitly control the mapping of threads and memory to the indivual sockets and cores, will never achieve top performance on a 32 core NUMA system. Most benchmark utilities are not written in this way, which regularily might people belief that the single socket system gives more bang for the buck. This is correct if the assessement is done with these kind of software, it is not correct with properly developed software.

This is especially true for memory bandwidth performance checks .Your decline might come from the utility's inability to properly understand your topology. Check with stream benchmark. A system with 1600MHz memory should be in the 80 GB/sec range

7)
Q-Code 22: haven't checked with the Z9PE-D8, but the Z9PE-D16 displays 22 as normal system value in operation

General questions:
Have you really checked that your power supply is delivering stable performance on the 12V CPU socket supplies? Did you change the connectors? The frequent uncoordinated reboots could lead into this direction.
Just to make sure I asked: Do you use retail versions of E5-2687W or engineering samples (ES)? If the latter, many of the issues could be from this choice.

Andy


----------



## TheBlackHole

I would first like to thank you very much for your post - really complete, competent and accurate - in the suggestions and explanations. Thank you for your valuable time that you gave me! Below I have highlighted my answers / comments to what you wrote.
Quote:


> Originally Posted by *AndyE*
> 
> I do have 2 ASUS systems Z9PE-D16 and Z9PE-D8 with 2x E5-2660 and 2xE5-2687W which i regularily change in between. Main memory is 256GB and 128 GB respectively, ECC 1600MHz.
> 
> To your questions:
> 1) The Q-Codes are listed in the manual and provide at least a solution to know until when the POST run ok. Good that the system is now starting
> A good summary for initial boot issues:
> http://www.tomshardware.co.uk/forum/261145-31-perform-steps-posting-post-boot-video-problems
> 
> *Thanks. Unfortunately the manual does not always provide detailed explanations about the codes, and many of them are not even listed.....
> 
> 
> 
> 
> 
> 
> 
> *
> 
> 2) If memory is "missing" then the first thing I would do is to reset the CMOS memory. Potentially, in trying to get the system starting, you changed essential settings and impacted this part of POST
> 
> *Well, I changed a lot of settings, but only after every try @ default settings was useless. Anyway I resetted the CMOS in many occasions and this sometimes has solved the issues.*
> 
> 3) ME is the Intel Management Engine which is part of Intels vPro offering. Basically it is a small computer on the motherboard supervising the operation of the system, even if the main CPUs are turned off - i.e. you can remotely start the Asus MB via this mechanism
> 
> *Thank you!: I was not able to find on the Internet such a simple & clear explanation!*
> 
> 4) With default CMOS settings, the MB shows the tarbet base frequency, which is in your case 3100MHz. If perf mode and similar things are changed, it could be different (haven't personally tried it)
> 
> *As explained in my first post, the BIOS automatically sets the CPUs speed @1200 MHZ when I power up the system.
> While this is perfectly fine to me when the system is posting, I was wondering if under Windows enviroment it is still normal that the CPU's speed remains at 1200 MHZ @ idle.
> 
> 
> 
> 
> 
> 
> 
> 
> Of course when the CPUs are loaded with something to do, they increase the frequency till the maximum as expected.
> 
> 
> 
> 
> 
> 
> 
> *
> 
> 5) This is normaly behavior. Even if the workload of AIDA, HWInfo etc is not big, you are executing software on your system. There is no hard affinity between the executing thread in an application abnd the physical core in the socket - shows up as a kind of random pattern. Benchmark software potentiall wakes up the CPU to get the timing operation right. I would *only* be concerned, if the system with good benchmark software under load would not go to either 3100MHz or better 3400 MHz (depending on your cooler). 3400 MHZ is the max frequency for all cores in Turbo mode. For 1-2 core under load the speed can go up to 3800 MHz. Good benchmark software for such a system is Intel's Linpack, be more critical of the results of the usual known benchmark apps - they are not optimized for NUMA boards like yours.
> 
> *Ok, you should be right. The strange thing is that I initially had not noticed this behavior, and it seemed to me that HWInfo64 (for ex.) always showed the frequency of all cores at 1200 MHz (of course @ idle), without sudden surges in frequency & voltage and activation of the turbo as is currently the case.....
> 
> 
> 
> 
> 
> 
> 
> *
> 
> 6)
> Memory amount: This is normal
> Memory is used for all kind of things like I/O devices are mapped into physical memory space. Some utilities see one POV, some have other approaches. 48 MB difference look like a PCI express card is mapped into memory space.
> 
> *Thanks! I did not know this possibility, and this may be the case indeed! Sometime it shows 32720 MB, some other 32718MB but I cannot determine why it changes or I don't remember: maybe the times when I tryed to put the video card in others slots just to check system stability or attempting to solve issues.
> 
> The strange thing though is that if I put the sticks on socket 1 only (either 1 or all 4 sticks), every utility (also CPU-Z) shows the correct values without even a single "lost" MB.
> 
> Also the same happens if I put two sticks on socket1 and the others two on socket 2 but on DIMM "G" & H" instead of "E" & F" as the manual reccomend. What do you think? And, if I keep the sticks on the "wrong" DIMM ("G" & H") can I experience some instability or random issues/performance loss?
> 
> I did some tests and benchmarks: does not seem that there are substantial differences in performance if I put the sticks on the "wrong" DIMM, but maybe I need some deeper analysis. Please also note that I can even put all four sticks on socket 1 leaving CPU2 without memory on socket2 and still I get QuadChannel mode!
> 
> But I suspect that I can not use this setting....what do you think? This might damage the hardware? Or simply in some circumstances may occur performance loss due to the lack of memory on socket2? (QPI issues,etc.)*
> 
> 4 x 8 GB memory is a suboptimal configuration for this system. One of the reasons for a dual socket MB is a the increased bandwidth you might get. In this configuration the 16 cores of the 2xE5-2687W will starve for memory bandwidth to achive their regular performance. Basically, you run the 8 core socket with the memory bandwidth of a 4 core i7-3770. Either switch to 8x4 GB or 8x8GB.
> 
> *You are right. I was misled by an Asus "technician" who told me that I could get the "Quad Channel mode" with just 4 sticks and that it was preferable to use 8GB sticks instead of 4GB (for performace, OC,etc.).
> 
> However, this is true ONLY if you use a single CPU and then you put all the 4 sticks on socket1.
> 
> Evidently the "technician" did not know that with 2 CPUs things are different and you can not keep the "Quad Channe mode" anymore, having to distribute the 4 sticks on the two sockets
> 
> 
> 
> 
> 
> 
> 
> 
> 
> This way the system cannot go beyond "Dual Channel mode".
> 
> Also, if I put just one stick on socket1 and another one on socket2, they run in "Single Channel mode" - no matter the DIMM I chose to put them in.
> 
> 
> 
> 
> 
> 
> 
> Is this normal? I cannot achieve Dual Channel unless I put in all four sticks? I ask because this seems odd to me and I wonder if this may be a sign that something is wrong.....
> 
> 
> 
> 
> 
> 
> 
> *
> 
> To identify possible errors, I'd start with one populated socket, and go with 1 RAM DIMM and start there. Run the system under load for a few hours (linpack i.e.) and check the stability in its minimal config. Move from there by adding more memory Dimms to one socket. If all goes well, THEN add the second. If problems arise, change back to a single socket config with the "faulty" CPU and re-check stability. If stability is there, your setup might have been screwed for dual socket ops (or one CPU is indeed defect. Clear the CMOS and re-check with 2 sockets.
> 
> *I already did as you reccomend! Just 1 RAM stick, etc. All sort of bech, stress test,etc. The RAM seem absolutely ok. Anyway the system seems stable but I will try Linpack as you suggest.*
> 
> As said, many classic benchmark utilities are "confused" with NUMA systems or if the performance goes in regions beyond the design of the software. Some of them get actual slower (as data has to be moved via the QPI interconnect and there are WAY too many cores to choose from. Software which does not explicitly control the mapping of threads and memory to the indivual sockets and cores, will never achieve top performance on a 32 core NUMA system. Most benchmark utilities are not written in this way, which regularily might people belief that the single socket system gives more bang for the buck. This is correct if the assessement is done with these kind of software, it is not correct with properly developed software.
> 
> *Thanks for the explanation!!!*
> 
> This is especially true for memory bandwidth performance checks .Your decline might come from the utility's inability to properly understand your topology. Check with stream benchmark. A system with 1600MHz memory should be in the 80 GB/sec range
> 
> *Well, 80 GB/sec is true for system equipped with 64GB or more (like yours!).
> 
> Indeed, I read some reviews about a system like mine with 64GB where the bandwidth was 90 or more GB/sec.
> 
> In my system however, I have just 32GB and according to Sandra the bandwidth is around 45 GB/sec, which I think should be completely satisfactory - at least according to the researchs I've done (comparing performances on similar systems ,etc.)*
> 
> 7)
> Q-Code 22: haven't checked with the Z9PE-D8, but the Z9PE-D16 displays 22 as normal system value in operation
> 
> *Thanks! Yes, I found a couple of days ago on Asus Z9PE-D8 offcial forum a post where another user said he asked directly to the Asus technical service and they replied that "code 22" is absolutely normal when BIOS POST is done and Windows take over.
> 
> 
> 
> 
> 
> 
> 
> *
> 
> General questions:
> Have you really checked that your power supply is delivering stable performance on the 12V CPU socket supplies? Did you change the connectors? The frequent uncoordinated reboots could lead into this direction.
> Just to make sure I asked: Do you use retail versions of E5-2687W or engineering samples (ES)? If the latter, many of the issues could be from this choice.
> 
> *My PSU is one of the best (I personally think the best) available on the market in terms of quality and stability (among other things): of course I checked the voltages regulations and they are rock solid, with a fluctuation of a mere 0,001-0,005 V on CPUs.
> 
> All the others voltages are "paralyzed" (BIOS monitor).
> 
> My CPUs are retail.*
> 
> *P.S.: which BIOS version do you use?
> *


*THANKS!!!*


----------



## AndyE

Quote:


> Originally Posted by *TheBlackHole*
> 
> I would first like to thank you very much for your post - really complete, competent and accurate - in the suggestions and explanations. Thank you for your valuable time that you gave me! Below I have highlighted my answers / comments to what you wrote.
> *THANKS!!!*


You are welcome.

I got my first dual SB system a few months ago and I am still learning how to better optimize it for different workloads - more on that later.

Some further comments to your points:

1) After I wrote my initial comment on your original post I saw that you tried multiple approaches. #1 priority in testing is to get reproducible errors. If errors aren't reproducible, then they can't be fixed reliably (exceptions apply)

2) ok
3) ok
4) I would not get too deep on this representation. The important thing is: Does your system perform as expected when under load and does it save energy when there is no load, Everything in between isn't black and white anymore, there are many shades of gray possible. If Windows is reporting 1200 MHZ at idle than I would consider this normal (my system does this as well). BTW, if your don't have one yet, try to get one of those cheap power meters to see what the power consumption of your system is.With 2 CPUs and 8 DIMMs, a good PSU but no PCI cards plugged in, the power consumption at the wall outlet should be well below 100 watt when the system is idle. HWInfo64's display of the 8 cores (The scket has to be selected) has usually this kind of sporadic peaks to 3100, 3400 or up to 3800 MHz. Any modern operating system has tens of processes working in the background. Usually dormant, those services wake up and do some processing. Nothing serious, just housekeeping stuff. From an energy optimization perspective it is better for the OS to crank up the frequency of the respective core for this work to be finished asap to get the system earlier down back in low power mode vs. running these tasks at 1200 MHz.

5+6) As said, different utilities might use different views. If the very same utility reports different mem sizes without any HW change than this would be suspect, everything else is rather a SW dependency how some data is being read and interpreted. An important table during boot-up is the SRAT (System Resource Affinity Table) which describes in pedantic detail the topology of the system so that operating systems like Windows or Linux can understand what the architecture of this system is and adjust appropriately during startup.

On your benchmarking exercise and not seeing a huge difference:
Beware of benchmarks. The "Art" of benchmarking is way more complex than most benchmarking communities go into. Some points:

a) Does this particular benchmark reflect the applications you are going to use? It doesn't make sense to see the 330 GFlops of floating point performance your dual E5-2687W are capable of if you will never run an app with floating point data

b) What is the memory footprint and the memory access pattern of the benchmark? If your benchmark checks the CPU datapath only, then it might fit into level-1 (and potentially level-2) cache hierarchy to exploit all the speed you can get. If the benchmark needs more memory - let's say 16 MB - then all can fit in level-3 cache. BTW, level 3 cache is managed by the uncore part of your E5-2687W, level 1+2 is managed by the respective cores. Extending this model further: As long as all these benchmark fit into your cache, then the memory architecture, configuration and performance is more or less irrelevant. It isn't used anyway.

c) If you take an app with a significant memory footprint (i.e 4 GB), the config of your memory becomes more likely a critical component for good performance. A good paper for SB memory configuration is here. It is written by Fujitsu, but applies to your Asus MB likewise (link)

d) benchmarking is at an inflection point. Until now, most performance improvements came from ILP (instruction level parallelism) and frequency gains. This gave the SW community for many decades a free ride to take benefit of increased performance without touching their software. This period is over. Going forward applications have to know and need to be explicitly programmed to leverage further increases via data parallel, task parallel and thread parallel approaches. Apps which haven't been designed in that way, won't show what your system is really capable of. So, say goodby to many familiar benchmarks and say welcome to the world of parallelism.

e) Your point on QPI: Depending on the type of memory allocation of the application and the memory access pattern, the QPI link might only be used to send infrequent status updates from one half of the system to the other or it becomes a critical bottleneck if the memory access pattern requires frequent access from memory connected to the other socket (i.e. when sorting data). If your applications has a kind of "pulse" pattern for the QPI, your system might benefit that you turn off energy savings mode for the QPI interconnect (wake up times are relatively long)

f) Your system is capable of supporting 8 memory channels and the minimum memory size for full memory performance with 1600 MHz Dimms is 16 GB (i.e. 8 x 2GB Kingston ECC DDR3 for 25 USD each). With this config your memory speed is in the 80 GB/sec range (measured with STREAM). The reason you will find higher transfer rates in some apps is the difference in memory access patterns. Depending on the memory pattern the applications use to access the Dimms, many optimizations are possible (rank interlave, dimm interleave, page access, sequential). These are independent (complementary) to the CL settings many here are optimizing their systems around. Even with the best CL times (i.e. 5-5-5-10) data transfer speed is well below its optimal speed if the application has the wrong access pattern. With a 1600 MHz Dimm, the theoretical upper bound is 12.8 GB/s. Expect everything from 3 to 11 GB/s on practical performance - the application design is much more relevant than CL optimizations.

7) ok

General comments)
PSU: In an attempt to identify an error or misbehavior, you are quite fast in excluding potential causes.
a) Your sample might be faulty
b) the cable from the PSU to the MB might have a contact issue
c) the plugs of these cables on the PSU or MB might have some transient issues
d) etc.
The reason I write this is that it happened to me a while ago. My PSU (Enermax 1500 W) was working fine, but one of the CPU 12V connectors didn't fit perfectly on the MB. Consequence: many irregular reboots of the system, under load, without load, etc ...

I am using the latest BIOS on both boards

cheers,
Andy


----------



## motogplvr

Do you guys think two of these would work on the Z9PE-D16/2L with the latest bios?


----------



## TheBlademaster01

2 of what? I know mose B0 work on the non .cap bioses of the Asus boards including the server board you listed and I believe Z9PA-D8 as well.


----------



## AndyE

Quote:


> Originally Posted by *motogplvr*
> 
> Do you guys think two of these would work on the Z9PE-D16/2L with the latest bios?


If you refer to the E5-2687W's ? I am running them on a Z9PE-D16 (the one with 4 x GBit LAN ports)


----------



## lloyd mcclendon

**** BIOS UPDATE RELEASED *****

well .. 4 days ago

https://www.asus.com/Motherboards/Z9PED8_WS/#support_Download_36
Quote:


> Version 3506
> Description Z9PE-D8 WS Formal BIOS 3506 release
> 1. Fix the issue of that system can't boot to OS after using HighPoint 2720/4320 RAID card to build RAID.
> 2. Patch NVIDIA VGA card, K5000.
> 3. Add Network Stack related options in BIOS setup.
> 4. Fix FAN contorl abnormal after resume from S3.
> 
> [Warning]
> *If your BIOS version is 0703 or older: Go to BIOS-Utilities to download and flash the BIOS to version 3109 firstly before updating the BIOS.
> File Size 3,9 MBytesupdate *2013.05.07*
> Download from Global


YES .. hopefully this fixes some of the issues we've been having. I hope my resume problem is fixed in #4 .. darn FAN contorl LOL







.. I may also be indirectly affected by the problem of #2 where it doesn't initialize the VGA space properly. After a few months of testing, even for me, I've found that occasionally either the GTX570 OR the AMD 7870 isn't detected on boot.









I guess I'll be the guinea pig. Anyone else that is brave enough, please try and report back the good or bad. I've been waiting for this for such a long, long time, I really hope I can go forward with better stability. It's ironic that I was over looking at the supermicro boards, then said "I really just need a good bios update for the z9" .. checked again and finally it was there ..


----------



## motogplvr

Quote:


> Originally Posted by *TheBlademaster01*
> 
> 2 of what? I know mose B0 work on the non .cap bioses of the Asus boards including the server board you listed and I believe Z9PA-D8 as well.


sorry my link didn't work...

http://www.ebay.com/itm/290910828818

This auction states they have tested these with the Z9PE-D16/2L and .cap bios 3401 (see pictures in the auction). I sent them a message and they verified this is the case, so it seems that these B0 QA8X ES cpus will work with the .cap bios.

I bought two and a Z9PE-D16/2L so hopefully everything goes smoothly.


----------



## lloyd mcclendon

well I've flashed the bios, luckily it did not brick the system and it still boots normally.









however, suspend / resume still does not work





















, my usb keyboard still does not work in the bios









unsure if the vga card detection problem is resolved or not, time will tell


----------



## TheBlademaster01

Which board and which CPUs?


----------



## TheBlademaster01

I have the board but only a Corsair TX750. Can you power the board with a single EPS connector and the 4 pin molex or do you need to power the second EPS as well (what is the function of the 4pin plug actually?). Also are there molex to EPS cables?


----------



## ZDngrfld

Quote:


> Originally Posted by *TheBlademaster01*
> 
> I have the board but only a Corsair TX750. Can you power the board with a single EPS connector and the 4 pin molex or do you need to power the second EPS as well (what is the function of the 4pin plug actually?). Also are there molex to EPS cables?


Pretty sure that's to power multiple GPU setups. If it's anything like their other motherboards, that's what it is. I don't have it plugged in on mine. You *HAVE* to have both EPS connectors plugged in to even have this board boot with one CPU. I think your only option, aside from buying a new PSU, is to use a Molex/PCI-e to EPS adapter.


----------



## TheBlademaster01

Yeah was looking into that, I just noticed how no one ever mentioned anything about the 4pin molex on the board lol. Then there were some YT video reviews on the board with dudes spreading misinformation and he told you could "replace" the second EPS plug with the 4pin if the PSU didn't have it (he had a british accent).

Thanks though, +REP


----------



## lloyd mcclendon

Even the ASUS tech I spoke with didn't know what that connector was really for. The manual of course has about 3 words about it and it's very unclear. I've always had it plugged in, but would like to know what it's for and what the difference would be.

And yes, you do need both 8pins even with one CPU. I've tried, it doesn't get very far









ALSO there is a jumper in between the PCI slots, something like TESLA_SM_BUS or something. It says it changes something from PCH to BCM. Any ideas what that is for???









BTW the bios update also did not fix the "card sometimes undetected" issue either, it does nothing. Disappointed


----------



## nhcoohrh

Currently I'm running an Asus Z9pe-d8 ws.

And seams only 1 SLOT detects GPUs, the SLOT 3.

All other PCI-E slots doesn't detect the GPU, even tried crossfire and was like it was not there.

Tried two different HD 7950 (Asus and HIS).

All updated to latest firmware of BIOS.

Should I do RMA of the Board?

Or this a question of AMD model, and should try another model.


----------



## garuda

I found this thread and thought my questions are best suited for this thread since it likely attracts Z9PE-D8/WS owners.

1. Does anyone know of any fan speed monitoring software that works with an ASUS Z9PE-D8 WS workstation dual mobo with C602 chipset? I've tried most of the more popular monitoring software, but none seem to have the capability to monitor (vs. control) the speeds of the CPU and chassis fans connected to mobo headers.

2. For those who have the ASUS Z9PE-D8 dual board (or one with a C602 chipset), how do you monitor your fan speeds while in Windows mode?

3. Does anyone have any insights into whether ASUS integrated an "embedded" fan controller on mobo, which cannot be accessed by developers of third-party "fan monitoring software?" ASUS tech support seems to avoid addressing this question.

Help will be appreciated!


----------



## Migonacci

This is probably the wrong place to post this, but any photoshop users experience a trial reset of their adobe software? There seems to be an issue with this for x64 windows users but having reformatted several times on my machine with a barebone win 7 install + drivers, I can't figure out what is causing the adobe licensing to be reset. I've installed CS6 on several other x64 machines with same components (sans motherboard) and haven't run into licensing issue with CS6 suite.

This is a stretch but is there any possibility that this motherboard driver's can be causing some issues with the OS thats affecting CS6? This is really an anomaly for me since I haven't been able to replicate this behavior in any other machine.

adobe thread for the issue with no solutions
http://forums.adobe.com/thread/1114727?start=520&tstart=0

Sorry for the off topic post.


----------



## motogplvr

Quote:


> Originally Posted by *motogplvr*
> 
> sorry my link didn't work...
> 
> http://www.ebay.com/itm/290910828818
> 
> This auction states they have tested these with the Z9PE-D16/2L and .cap bios 3401 (see pictures in the auction). I sent them a message and they verified this is the case, so it seems that these B0 QA8X ES cpus will work with the .cap bios.
> 
> I bought two and a Z9PE-D16/2L so hopefully everything goes smoothly.


Well I just got back from a work trip and started assembling the system. This sucks I can't get the board to POST with these B0 stepping CPUs installed! This is a brand new D16/2L and the CPUs I linked in the thread. I have both EPS12v connectors and the 24 pin connected, tried only ram in the A1 slot (as the manual says), took out CPU2 and only have CPU1 installed now. I have tried an add-in PCIE card and also tried the onboard video. Nothing works! The onboard LED readout just stays at 00 and never changes and the board cycles power to itself in a loop every 6 seconds or so...

THIS SUCKS! I think my days of messing with ES CPUs is over after this and another attempted 1356 dual CPU ES build.

Anyone have any ideas?


----------



## TheBlademaster01

It could also be the board. I know someone who had to RMA the D16/2L 4 times...


----------



## joetesta

Hello -
my coworker and i are trying to build a system with this Z9PE-D8 and it will not POST, no Q-code visible. The power LED and reset LED come on. When we hit power, the CPU fan spins for 1 second then stops
2 X Xeon E5-2630
4 x 8GB CORSAIR 240-Pin DDR3 SDRAM DDR3 1333
PSU: CORSAIR HX Series HX750 750W

ASUS suggested RMA to our vendor but we already returned one for the exact same problem and it feels like it's not the board. At this point we believe the PS may be insufficient, we used an EPS/ATX 4+4 to connect the 2nd 8-port CPU power.
We have it on the bench with a single CPU and single stick of RAM and the symptoms are always the same - no POST, brief fan spin, no Q-code. No sound from the (questionable?) little speaker we hooked up.
Questions: Should it POST with no cooler attached to the CPU?
There is a VGA video connector on the board and a cable it came with that we attached, should that work? (ASUS support claimed there's no onboard video)
We weren't planning to use GPU boost. Will a cheap PCI-e video card work?

Thank you very much in advance, really looking forward to seeing the bios someday.
cheers,
Joe


----------



## ZDngrfld

Quote:


> Originally Posted by *joetesta*
> 
> Hello -
> my coworker and i are trying to build a system with this Z9PE-D8 and it will not POST, no Q-code visible. The power LED and reset LED come on. When we hit power, the CPU fan spins for 1 second then stops
> 2 X Xeon E5-2630
> 4 x 8GB CORSAIR 240-Pin DDR3 SDRAM DDR3 1333
> PSU: CORSAIR HX Series HX750 750W
> 
> ASUS suggested RMA to our vendor but we already returned one for the exact same problem and it feels like it's not the board. At this point we believe the PS may be insufficient, we used an EPS/ATX 4+4 to connect the 2nd 8-port CPU power.
> We have it on the bench with a single CPU and single stick of RAM and the symptoms are always the same - no POST, brief fan spin, no Q-code. No sound from the (questionable?) little speaker we hooked up.
> Questions: Should it POST with no cooler attached to the CPU?
> There is a VGA video connector on the board and a cable it came with that we attached, should that work? (ASUS support claimed there's no onboard video)
> We weren't planning to use GPU boost. Will a cheap PCI-e video card work?
> 
> Thank you very much in advance, really looking forward to seeing the bios someday.
> cheers,
> Joe


First off... Don't ever try to boot a system without a heatsink... You'll burn up a processor really fast that way. Have you been running the processors without them? I sure hope not...

That power supply is fine.
A low end PCI-e video card works fine. That's what's in mine.

I had the same issue with mine. No QCodes, brief fan spin up. I had to RMA the board. Has been working great since then


----------



## joetesta

> First off... Don't ever try to boot a system without a heatsink... You'll burn up a processor really fast that way. Have you been running the processors without them? I sure hope not...

It has never powered on and ran. The first board we built completely including the coolers, it had the same exact problem (no POST, immediate shutdown) and the heat sinks are a massive pain to put on and take off. They are "COOLER MASTER GeminII S524 120mm Long Life Sleeve CPU Coolers" and 2 of the screws are under the fan and take a decent amount of pressure to attach.
We sent that first board back 1 week ago and received this one on Wednesday. Before we even put the heat sinks on the second board, we'd figure to make sure it POST's to the bios. But since it never powered on or gave a Q-code, we didn't attach the heat sinks and we closely monitored the CPU for a rise in temperature that never happened at all. We wanted to test each CPU in socket 1 and they both acted the same. If they fried without us even feeling any warmth, i'll eat my socks.

So based on your experience it does sound as if we need to RMA the board again....

What's up with that onboard VGA plug and the cable that goes with it; should we expect it to work on boot or does it need drivers?

thank you,
Joe


----------



## BlackenedTush

joetesta,

Yeah ZDngrfld is right... you've got a capable PSU.

I could have misunderstood your description; but it sounds to me like you are powering the board with:

one 24-pin ATX PWR cable,
one 8-PIN EPS12V CPU cable (plugged into one of the 4+4 CPU jacks)
a second 8-PIN EPS12V CPU cable (plugged into one of the 4+4 CPU jacks)
That setup should work. But if you only have one 8-pin 12v connection, you will get exactly the symptoms that you are describing.

Now from what I understand in the product description of your PSU, it only comes with one 8-PIN EPS12V CPU Cable.
What are you using for the second?


----------



## BlackenedTush

Migonacci,
I'm using both Photoshop CS5 & Photoshop CS6 (CC) on this mobo, with Windows 7 (x64) Professional SP1...
Early on in the life of this system I had some bad RAM. So Photoshop CS5 got corrupted along with Windows and some other apps...
But after replacing the RAM and reinstalling everything, I've never had this or any other problem.
If I ever find out something or think of something, I'll try to remember to message you. Every little clue helps.
Good luck.
BT


----------



## joetesta

thanks for your help B.T.
the second 8 pin is connected with this modular cable that plugs into the power supply:
http://www.newegg.com/Product/Product.aspx?Item=N82E16812162018
Silverstone Model PP06B-EPS55 21.65" (550mm) Sleeved EPS/ATX12V 8pin Cable


----------



## ZDngrfld

Quote:


> Originally Posted by *joetesta*
> 
> thanks for your help B.T.
> the second 8 pin is connected with this modular cable that plugs into the power supply:
> http://www.newegg.com/Product/Product.aspx?Item=N82E16812162018
> Silverstone Model PP06B-EPS55 21.65" (550mm) Sleeved EPS/ATX12V 8pin Cable


Maybe that cable doesn't have the proper pinout to be compatible with that power supply. Startech makes an adapter that uses PCI-e power


----------



## joetesta

Thanks ZDngr,
Based on the comments in the Asus support forum, starting April 2012 several users reporting the same issue, we decided to try a different power supply, we're planning to have one of these next week
http://www.newegg.com/Product/Product.aspx?Item=N82E16817151109
SeaSonic X-1250 1250W ATX12V / EPS12V SLI Ready 80 PLUS GOLD Certified Full Modular Active PFC Power Supply

although i'm sure the adapter would be a bit less expensive








It was before I'd posted here and I think it was already ordered, so we'll see what happens


----------



## ZDngrfld

Quote:


> Originally Posted by *joetesta*
> 
> Thanks ZDngr,
> Based on the comments in the Asus support forum, starting April 2012 several users reporting the same issue, we decided to try a different power supply, we're planning to have one of these next week
> http://www.newegg.com/Product/Product.aspx?Item=N82E16817151109
> SeaSonic X-1250 1250W ATX12V / EPS12V SLI Ready 80 PLUS GOLD Certified Full Modular Active PFC Power Supply
> 
> although i'm sure the adapter would be a bit less expensive
> 
> 
> 
> 
> 
> 
> 
> 
> It was before I'd posted here and I think it was already ordered, so we'll see what happens


Just so you know, my 2P E5-2650 using this board pulls 250 watts from the wall while using 100% CPU, so you're going beyond overkill with that PSU. I'm using a Seasonic X750 in that rig. There's nothing wrong with overkill, though.


----------



## garuda

I can't enter the BIOS settings utility (using DEL key) unless all of the USB 3.0 devices are disconnected from USB-3 ports. Does anyone else w/ Z9PE-D8 board have this issue?

Also Joe, I'm not sure, but I think VGA needs drivers (unless embedded in BIOS). And I hope your new replacement board works; pretty frustrating when you can't even get off the launch pad.









PS: be careful not to over-tighten the 2011 mounting plate screws. Some have reported it has warped the board and prevented POST/boot.


----------



## joetesta

Great news, we hooked up the new SeaSonic X-1250 1250W power supply and the board boots just fine, we're updating bios now.
So I don't know whether the cable adapter was incompatible or what was wrong with the other 750W power supply, but we are making progress


----------



## TheBlademaster01

Quote:


> Originally Posted by *joetesta*
> 
> Great news, we hooked up the new SeaSonic X-1250 1250W power supply and the board boots just fine, we're updating bios now.
> So I don't know whether the cable adapter was incompatible or what was wrong with the other 750W power supply, but we are making progress


Thanks for testing, I was planning to use an adapter on my Corsair TX750 but you just saved me a lot of effort









+REP


----------



## ZDngrfld

Quote:


> Originally Posted by *joetesta*
> 
> Great news, we hooked up the new SeaSonic X-1250 1250W power supply and the board boots just fine, we're updating bios now.
> So I don't know whether the cable adapter was incompatible or what was wrong with the other 750W power supply, but we are making progress


Cool! I'm glad you got it working.
Quote:


> Originally Posted by *TheBlademaster01*
> 
> Thanks for testing, I was planning to use an adapter on my Corsair TX750 but you just saved me a lot of effort
> 
> 
> 
> 
> 
> 
> 
> 
> 
> +REP


Well, he was using a Silverstone cable. It states in the product description that it's "Compatible with all SilverStone modular PSU released after 2009". So I bet it would have worked on a Silverstone PSU. If you use an actual adapter like one of these, it should work fine.


----------



## Migonacci

Quote:


> Originally Posted by *BlackenedTush*
> 
> Migonacci,
> I'm using both Photoshop CS5 & Photoshop CS6 (CC) on this mobo, with Windows 7 (x64) Professional SP1...
> Early on in the life of this system I had some bad RAM. So Photoshop CS5 got corrupted along with Windows and some other apps...
> But after replacing the RAM and reinstalling everything, I've never had this or any other problem.
> If I ever find out something or think of something, I'll try to remember to message you. Every little clue helps.
> Good luck.
> BT


Thanks for the info blackenedtush. I think I may have isolated the problem. It seems like the license gets reset whenever I put the computer to sleep and wake from it; I may have to change the sleep state in the bios, but I'm going to do some more troubleshooting just to make sure.


----------



## BlackenedTush

garuda,
Depending on the design of the cooler (whether there's a supporting back plate etc.), and the design of the mobo, having the CPU cooler screws too tight will definitely cause problems. One common symptom is that it can prevent or intermittently disrupt your GPU from driving the monitor, so even if it posts, you won't be seeing the fruits of your labor... only a message telling you that there is no HDMI signal.


----------



## BlackenedTush

Migonacci,

Hey, glad to hear there's some light at the end of the tunnel!

It seems to me that hibernation and sleep states frequently end up being on the list of bad actors in any complicated system I've had.
And on this build, they're again on the _bad-actor list_; but for a different reason... I have an SSD boot drive which only stores the OS & apps... and like most SSDs these days, high _cost/GB_ still limits their size.

I have done the typical things one does to optimize SSDs.
One of those things has to do with managing what Windows puts on the system disk. And when it comes to sleep and hibernate, I'm not potentially dealing with the Adobe issues that you are because I've disabled both of them. Since my machine has 64GB of RAM, Windows 7 would automatically reserve 64GBs of my 120GB SSD as untouchable space to store the system state whenever it hibernates. That's over half the capacity! I can't afford that.

BT


----------



## garuda

Hi guys,
I'm working out the minor glitches in my Z9PE, and BT encouraged input & questions. So&#8230;. Is anyone having issues with USB 3.0? I've heard that this board has issues w/ USB3, but weren't specific. Admittedly, this is minor, but I can't launch into BIOS settings (@ POST time) when USB 3.0 devices are attached to rear USB3 ports. I must disconnect all USB-3 cables in order to launch BIOS settings utility. Do you guys experience the same?

And I couldn't get my Uspeed/Anker USB-3 expansion card to work in PCIe slot#4. The card ports were dead and the Device Mgr have an "Exclamation Point (!) under the USB Ctlr grouping (USB3 Root Hub) at bottom of listing. ASUS tech support had no idea why.


----------



## anubis1127

I got my Z9 board back from Asus RMA, and now everything works great. First one I got was just a dud apparently. It's on an ancient BIOS rev from last year, but I figure since everything's working for my purposes right now, I better not mess with it.


----------



## TheBlademaster01




----------



## anubis1127

Hi BM!


----------



## TheBlademaster01

Still haven't got all the parts together









Parts aren't cheap at all. Need a water block and coolant now.


----------



## Donkey1514

Quote:


> Originally Posted by *TheBlademaster01*


Quote:


> Originally Posted by *anubis1127*
> 
> Hi BM!


----------



## anubis1127

Quote:


> Originally Posted by *TheBlademaster01*
> 
> Still haven't got all the parts together
> 
> 
> 
> 
> 
> 
> 
> 
> 
> Parts aren't cheap at all. Need a water block and coolant now.


No, they aren't. I still need a case for mine, haha, it's sitting on some mobo standoffs on an old Antec case I layed down horizontally. A better PSU, some matching heatsinks would be nice too, right now I'm using a Scythe Mugen2, and H80 on it, LOL. I swear I have the most ghetto 2P on OCN.


----------



## TheBlademaster01

Quote:


> Originally Posted by *Donkey1514*












Do you miss your board Donkey







?
Quote:


> Originally Posted by *anubis1127*
> 
> No, they aren't. I still need a case for mine, haha, it's sitting on some mobo standoffs on an old Antec case I layed down horizontally. A better PSU, some matching heatsinks would be nice too, right now I'm using a Scythe Mugen2, and H80 on it, LOL. I swear I have the most ghetto 2P on OCN.


This is the state of my system










Spoiler: Warning: Spoiler!







Going to rip out the 2600K + Z77 and get it in a mainstream case like it's supposed to and get the 2P in the Raven. Apparently Intel agrees that my 2600K will still be okay for gaming by confirming it with releasing Haswell lol


----------



## anubis1127

Haha, very nice, that's going to be a nice looking rig once you've completed it.


----------



## Donkey1514

Quote:


> Originally Posted by *TheBlademaster01*
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> Do you miss your board Donkey
> 
> 
> 
> 
> 
> 
> 
> ?
> This is the state of my system
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> Spoiler: Warning: Spoiler!
> 
> 
> 
> 
> 
> 
> 
> Going to rip out the 2600K + Z77 and get it in a mainstream case like it's supposed to and get the 2P in the Raven. Apparently Intel agrees that my 2600K will still be okay for gaming by confirming it with releasing Haswell lol


Kind of miss it but enjoy my SLI gtx690s more

That Corsair memory in that board =









Thar board would look really good with some EK nickle/plexi memory and cpu blocks with some Mayhem's pastel blue fluid


----------



## TheBlademaster01

I was planning to get Mayhems Pastel White and make it a very light shade of grey. I do have 3/4 liter of Pastel blue laying around but I want to prevent too much blue. Had too much blue builds before









I need a GTX Titan ish cooler or 690 in there ass well for maximum silver accents. I was planning to go with plexi nickel EK blocks in there


----------



## anubis1127

Quote:


> Originally Posted by *TheBlademaster01*
> 
> I was planning to get Mayhems Pastel White and make it a very light shade of grey. I do have 3/4 liter of Pastel blue laying around but I want to prevent too much blue. Had too much blue builds before
> 
> 
> 
> 
> 
> 
> 
> 
> 
> I need a GTX Titan ish cooler or 690 in there ass well for maximum silver accents. I was planning to go with plexi nickel EK blocks in there


It's too bad this: http://www.ekwb.com/shop/ek-fb-kit-asus-z9pe-d8-acetal-nickel.html doesn't come in a plexi + nickel option, that would look really good with your build.


----------



## TheBlademaster01

Yeah, that's the reason I'm only cooling the CPUs


----------



## lloyd mcclendon

does anyone here use suspend to RAM with this board? Does it always work or not? Which OS?

If I can't get this thing to resume it's going on ebay, I'm about tired of messing with it.







I may have to actually build a windows install to boot shudder to see if that OS can resume it.


----------



## joetesta

Greetings,
I wanted to note that on at least two occasions our system failed to boot with the Q-code stalled on 22 and a blinking white cursor on the screen. Each time a power cycle got it to boot normally on the next attempt. I noticed a few others reporting this Q-code, not sure if we should be concerned since it's allegedly a normal boot code;
Quote:


> Yes, I found a couple of days ago on Asus Z9PE-D8 offcial forum a post where another user said he asked directly to the Asus technical service and they replied that "code 22" is absolutely normal when BIOS POST is done and Windows take over.


maybe there are some issue communicating with the HDD where the OS is?


----------



## BlackenedTush

Hey anubis1127,

I haven't seen your name around here for ages.
It's great to hear your board is finally running!
Are you still using those ES chips, the tech tray, and on-board video?

BT


----------



## BlackenedTush

lloyd,
Sorry about your issue. You've been dealing with an awful lot with this board. I don't personally suspend to RAM or to disk and I'm not able to risk testing it at this time.
BT


----------



## anubis1127

Quote:


> Originally Posted by *BlackenedTush*
> 
> Hey anubis1127,
> 
> I haven't seen your name around here for ages.
> It's great to hear your board is finally running!
> Are you still using those ES chips, the tech tray, and on-board video?
> 
> BT


Hey BT,

That I am! I haven't ordered the tech tray I wanted yet, so I've just crafted my own solution out of an old case, and some motherboard standoffs for now.

The on-board video adapter I just found in the box I got from RMA yesterday, plugged it in, and display is all distorted. I don't really need display, so it's fine for now, probably just need to install drivers for it. I'm running Linux without a GUI, and the CLI doesn't even display correctly, LOL.


----------



## ViperJPB

Hey All. Sad to see so many people had issues with this board!? I have had none and it has all worked wonderfully. 2 E5-2687Ws, 32GB 1866 DDR3 (running in 1600 for the moment...had them functioning in 1866 but wouldn't pass prime95 after a few hours) 3x GTX 680 4GB cards (2 in SLI), Raid card, Black Magic Card, 2 SSDs, 9 internal hard drives including 8 in a SAS RAID 5 attached to the raid card. My only issues is keeping heat and noise in check. Also 32GBs is so not enough ram to use all the cores in an After Effects render....may need to move to at least 128GB soon.

Geekbench: 42,545
Sandra 2012: 31.99kPt (Was 9th place)
Cinebench: 25.45 CPU and 49.8 GPU
3DMark11: P 16,674 (2 680 4GB in SLI with a 3rd as PhysX)

My question for everyone has been mentioned earlier in this thread and I didn't see a clear answer or explanation.

QPI settings in the BIOS, there are items for Link0s, Link0p, and Link1. For some reason the first two are Disabled by default and the third one is Enabled. The manual doesn't explain what the are, any ideas?


----------



## lloyd mcclendon

Yeah, I would love to know also, I've messed around with those but did not notice any difference one way or the other, though I suspect it would have a big impact. There's a whole bunch of settings in there that the manual says 2 words about. ASUS doesn't know if you ask...







There's also that PCH/BCM tesla jumper I have no idea what it is.

Glad your system is working well .. since you have a stable build if you want to do a science experiment and try S3 suspend / resume and let us know if it actually works or not


----------



## ViperJPB

Yeah the manual is super vague in a lot of places. I am currently in the middle of a 6 month long project with the machine rendering almost around the clock. Otherwise I would totally be up for experimenting with those QPI link settings and running some benchmarks to see if it changes anything. I currently have no use for S3 suspend on my workstation, but when I am clear of some of these projects I'm working on now I will have to run that experiment for ya.

I'm curious too...as far as I can tell my GTX 680s are running at full speed but in PCIe 2.0...any way to force them into PCIe 3.0? I would also like to be able to do this for my RAID card specifically. I have seen reviewers getting faster transfer rates with the raid card in a PCIe 3.0 16x slot. Mine is in the 3rd 16x PCIe 3.0 slot...but as the cards are running at 2.0 I assume it is as well?


----------



## anubis1127

I too was unable to run my RAM at 1866, even though it is on the Asus Qualified Vendor list at its rated speed. I had to set it back down to 1600 for stability.


----------



## lloyd mcclendon

yep also stuck at 1600 here, same problem ... 2133 on QVL, 1866 won't post

I think if you messed with the settings enough (good luck) you might be able to get it [semi-]stable, but since the chips don't OC it may not be feasible.

We all really need a long Q&A chat with the engineers that know this board.


----------



## ViperJPB

Quote:


> Originally Posted by *anubis1127*
> 
> I too was unable to run my RAM at 1866, even though it is on the Asus Qualified Vendor list at its rated speed. I had to set it back down to 1600 for stability.


Yup mine was on the QVL as well. I got it at first glance stable with a voltage of 1.55 and all seemed well. Benchmarks were solid. But after a lengthy Prime95 it would give rounding errors. I never really tested it in real world work environments at that speed however. Again when this project is over I plan to play with it some more.

I have my noise down to acceptable levels....but the H60s are not the quietest things. Thinking about switching out the fans to double Noctua's or trying out the new slimmer NH-U12S heatsinks. Noctua tells me they are more than enough to cool these 2687Ws and they wont interfere with the top PCIe slot. However I'm worried with all the heat from other components that they may have issues. I read a german review where these were tested on this board and CPUs and were getting low 50s under load with Prime95!!! However they liquid cooled the second cpu and video cards so they could get un-effected results.


----------



## ViperJPB

Quote:


> Originally Posted by *lloyd mcclendon*
> 
> yep also stuck at 1600 here, same problem ... 2133 on QVL, 1866 won't post
> 
> I think if you messed with the settings enough (good luck) you might be able to get it [semi-]stable, but since the chips don't OC it may not be feasible.
> 
> We all really need a long Q&A chat with the engineers that know this board.


True dat


----------



## psyq3212

Quote:


> Originally Posted by *ViperJPB*
> 
> Yup mine was on the QVL as well. I got it at first glance stable with a voltage of 1.55 and all seemed well. Benchmarks were solid. But after a lengthy Prime95 it would give rounding errors. I never really tested it in real world work environments at that speed however. Again when this project is over I plan to play with it some more.
> 
> I have my noise down to acceptable levels....but the H60s are not the quietest things. Thinking about switching out the fans to double Noctua's or trying out the new slimmer NH-U12S heatsinks. Noctua tells me they are more than enough to cool these 2687Ws and they wont interfere with the top PCIe slot. However I'm worried with all the heat from other components that they may have issues. I read a german review where these were tested on this board and CPUs and were getting low 50s under load with Prime95!!! However they liquid cooled the second cpu and video cards so they could get un-effected results.


I am running Samsung M393B2G70BH0-YK0 (1600 MHz ECC 16 GB sticks validated at 1.35v @1600) at 1866 MHz and 1.55v, prime stable. I could run it @2133 MHz, too without changing of the timings but after several hours of heavy-duty use system throws machine check exception, meaning unrecoverable ECC error has occurred. I suppose loosening timings and bumping the voltage would help but I decided not to bother since I moved my algorithms to CUDA anyway.

Problem with 1866 MHz might not be due to memory alone - it could also be due to CPU's own IMC. Although Intel's processes are superb nowdays and most CPUs end up pretty fine when it comes to tolerances, it is still possible that your Xeons simply do not allow 1866 MHz DDR3 speed without bumping the VSa voltage.

You can try to bump VSa a bit. As for the DDR3 voltage, personally I would not go higher than 1.55v but for the sake of hypothesis validation I guess running a test with, say, 1.60v would not matter too much.

I am using Noctua NH-D14 (the big ones) and the clearance between the heatsink and PCIe x16 slot #1 is too small for a fan. However, I do not care too much at the moment since I am not able to use GTX TITAN when connected to PCIe slot #1 anyway (I strongly suspect PCIe link training failures).


----------



## anubis1127

Quote:


> Originally Posted by *psyq3212*
> 
> I am running Samsung M393B2G70BH0-YK0 (1600 MHz ECC 16 GB sticks validated at 1.35v @1600) at 1866 MHz and 1.55v, prime stable. I could run it @2133 MHz, too without changing of the timings but after several hours of heavy-duty use system throws machine check exception, meaning unrecoverable ECC error has occurred. I suppose loosening timings and bumping the voltage would help but I decided not to bother since I moved my algorithms to CUDA anyway.
> 
> Problem with 1866 MHz might not be due to memory alone - it could also be due to CPU's own IMC. Although Intel's processes are superb nowdays and most CPUs end up pretty fine when it comes to tolerances, it is still possible that your Xeons simply do not allow 1866 MHz DDR3 speed without bumping the VSa voltage.
> 
> You can try to bump VSa a bit. As for the DDR3 voltage, personally I would not go higher than 1.55v but for the sake of hypothesis validation I guess running a test with, say, 1.60v would not matter too much.
> 
> I am using Noctua NH-D14 (the big ones) and the clearance between the heatsink and PCIe x16 slot #1 is too small for a fan. However, I do not care too much at the moment since I am not able to use GTX TITAN when connected to PCIe slot #1 anyway (I strongly suspect PCIe link training failures).


Yeah, I haven't tried bumping up the voltage at all. I thought I was getting another kit of RAM to run in this board, but that fell through, so I may do some more playing around with the kit I have.

It probably doesn't hurt that you are running ECC too.


----------



## ViperJPB

Quote:


> Originally Posted by *psyq3212*
> 
> I am running Samsung M393B2G70BH0-YK0 (1600 MHz ECC 16 GB sticks validated at 1.35v @1600) at 1866 MHz and 1.55v, prime stable. I could run it @2133 MHz, too without changing of the timings but after several hours of heavy-duty use system throws machine check exception, meaning unrecoverable ECC error has occurred. I suppose loosening timings and bumping the voltage would help but I decided not to bother since I moved my algorithms to CUDA anyway.
> 
> Problem with 1866 MHz might not be due to memory alone - it could also be due to CPU's own IMC. Although Intel's processes are superb nowdays and most CPUs end up pretty fine when it comes to tolerances, it is still possible that your Xeons simply do not allow 1866 MHz DDR3 speed without bumping the VSa voltage.
> 
> You can try to bump VSa a bit. As for the DDR3 voltage, personally I would not go higher than 1.55v but for the sake of hypothesis validation I guess running a test with, say, 1.60v would not matter too much.
> 
> I am using Noctua NH-D14 (the big ones) and the clearance between the heatsink and PCIe x16 slot #1 is too small for a fan. However, I do not care too much at the moment since I am not able to use GTX TITAN when connected to PCIe slot #1 anyway (I strongly suspect PCIe link training failures).


Nice to know those sticks can get to those speeds. Especially since I plan to move up to 128GBs soon. I know 1866 vs 1600 really only shows in benchmarks and is not really reflected in the everyday 3D rendering or compositing. But still. Bragging rights. Are those sticks on the QVL? Haven't looked in awhile.

What's the issue with the Titan not running in the first slot? I have been thinking about moving from my three 680s to two Titans. Been running the first two blue 16x slots and last one with the 680s without a hitch.


----------



## psyq3212

Hmm - I cannot find any Samsung stick on the QVL list. Interesting.

As for the TITAN issues - I suppose it is a good old PCIe training. I suppose (but I do not have means to prove this) that when the GPU is in the PCIe slot #1, it unsuccessfully tries to establish a PCIe 3.0 x16 connection, and for some reason it fails.

When I plug the GPU in the PCIe slot #3, it works but the link is established as PCIe 2.0 x16 - I tried everything to make it work @PCIe 3.0 speeds but it is just impossible (and, yes, I did patch the registry).

So, finally, I gave up.

I noticed that a forum member had a post issue with B0 (ES1) Xeons and Z9PE-D16 - this is because ASUS removed support for B0 steppings at some point. I remember that the old (pre-cap) BIOS-es worked with ES1 samples. Most of the dual-socket boards nowdays do not support ES1 and ES2 (actually ES2/B1 never worked as it should) with the sole exception of Z9PE-D8 WS. Other boards will only work with C0/C1/C2 QS and C1/C2 retail steppings. The reason has nothing to do with the microcode, because all of the boards I saw have microcode update for the B0/B1 samples. Rather, the problem is in the QPI&MRC code where in the new versions probably they removed the errata for handling B0 quirks. Why the Z9PE-D8 work still, that's another question.

It will be interesting to see what's gonna happen when Ivy Bridge EP B0 samples start flooding the ebay


----------



## TheBlademaster01

What is the first .cap BIOS?

Really helpful post there. I have 2x B0 chips and a rev1.04 board that hasn't been flashed.


----------



## anubis1127

Quote:


> Originally Posted by *TheBlademaster01*
> 
> What is the first .cap BIOS?
> 
> Really helpful post there. I have 2x B0 chips and a rev1.04 board that hasn't been flashed.


3206 is the first .cap BIOS. I'm still using 0405 on mine, lol.


----------



## TheBlademaster01

Thanks anubis









Do you thnk it would have shipped with a board bought in November 2012?


----------



## derickwm

Who knows with Asus


----------



## anubis1127

It depends on how long the retailer had it probably, but I don't imagine it would have shipped with anything newer than 3109 (came out 9-2012), probably more likely 0405, or 0503.


----------



## TheBlademaster01

Well then, let's hope this system will boot









I've tested my RAM in Nardis yesterday and it was working. So many uncertainties lol. My PSU also misses the second EPS plug so I'm having a friend make a dual 4pin molex -> EPS adapter. There's also a Coolermaster Silent Pro M2 1.5kW PSU on sale for 125,- but I'm not sure if I should buy it


----------



## anubis1127

Quote:


> Originally Posted by *TheBlademaster01*
> 
> Well then, let's hope this system will boot
> 
> 
> 
> 
> 
> 
> 
> 
> 
> I've tested my RAM in Nardis yesterday and it was working. So many uncertainties lol. My PSU also misses the second EPS plug so I'm having a friend make a dual 4pin molex -> EPS adapter. There's also a Coolermaster Silent Pro M2 1.5kW PSU on sale for 125,- but I'm not sure if I should buy it


I have an updated BIOS chip, if you need an older one, I can keep my current one on 0405, and swap it with the newer one, and send you the one with 0405 too.


----------



## ViperJPB

Quote:


> Originally Posted by *psyq3212*
> 
> Hmm - I cannot find any Samsung stick on the QVL list. Interesting.
> 
> As for the TITAN issues - I suppose it is a good old PCIe training. I suppose (but I do not have means to prove this) that when the GPU is in the PCIe slot #1, it unsuccessfully tries to establish a PCIe 3.0 x16 connection, and for some reason it fails.
> 
> When I plug the GPU in the PCIe slot #3, it works but the link is established as PCIe 2.0 x16 - I tried everything to make it work @PCIe 3.0 speeds but it is just impossible (and, yes, I did patch the registry).
> 
> So, finally, I gave up.
> 
> I noticed that a forum member had a post issue with B0 (ES1) Xeons and Z9PE-D16 - this is because ASUS removed support for B0 steppings at some point. I remember that the old (pre-cap) BIOS-es worked with ES1 samples. Most of the dual-socket boards nowdays do not support ES1 and ES2 (actually ES2/B1 never worked as it should) with the sole exception of Z9PE-D8 WS. Other boards will only work with C0/C1/C2 QS and C1/C2 retail steppings. The reason has nothing to do with the microcode, because all of the boards I saw have microcode update for the B0/B1 samples. Rather, the problem is in the QPI&MRC code where in the new versions probably they removed the errata for handling B0 quirks. Why the Z9PE-D8 work still, that's another question.
> 
> It will be interesting to see what's gonna happen when Ivy Bridge EP B0 samples start flooding the ebay


Hmm...this is concerning. I bought this board for it's PCIe slot flexibility. Two Titans were my next move. So I can run them but just not in slot 1 and not in PCIe 3.0? This board was supposed to be 3.0 compliant. I have no issues with video cards in slot one so far. Of course running in PCIe 2.0.


----------



## ViperJPB

I had found a little freeware software patch earlier in the year that can force video cards to run at PCIe3.0....used it and had all my GTX 680s reading as PCIe 3.0 in GPU-Z. However I don't think these cards were meant for that and had some interesting little glitches so I undid it.


----------



## psyq3212

Quote:


> Originally Posted by *ViperJPB*
> 
> Hmm...this is concerning. I bought this board for it's PCIe slot flexibility. Two Titans were my next move. So I can run them but just not in slot 1 and not in PCIe 3.0? This board was supposed to be 3.0 compliant. I have no issues with video cards in slot one so far. Of course running in PCIe 2.0.


Your mileage might vary - there are people that had big issues with Z9PE-D8 WS board and getting Kepler-generation cards to work @PCIe 3.0.

I remember with older BIOS-es sometimes it was just plain impossible to boot 9 out of 10 times (!).

However, I've never seen PCIe 3.0 x16 speed on mine since I moved to Kepler









Please note that the PCIe 3.0 is connected to both GPU and the CPU. Pre-C2 Xeon E5 CPUs were not validated at all @PCIe 3.0, which means - it might work, or it might not. Fortunately for people who bought retail Xeon E5, large majority of them are C2 (there was an initial batch of C1-stepping Xeons with SR0xx stepping), but as far as I know these only went to key OEMs and partners pre-launch, so chances of having one are slim. However, ES/QS ones are a mixed bag - not officially validated, and anything can happen.


----------



## TheBlademaster01

Quote:


> Originally Posted by *anubis1127*
> 
> I have an updated BIOS chip, if you need an older one, I can keep my current one on 0405, and swap it with the newer one, and send you the one with 0405 too.


Nice, thanks man.

How do you change the bios IC, solder the 8 pins to the board?


----------



## ViperJPB

Quote:


> Originally Posted by *psyq3212*
> 
> Your mileage might vary - there are people that had big issues with Z9PE-D8 WS board and getting Kepler-generation cards to work @PCIe 3.0.
> 
> I remember with older BIOS-es sometimes it was just plain impossible to boot 9 out of 10 times (!).
> 
> However, I've never seen PCIe 3.0 x16 speed on mine since I moved to Kepler
> 
> 
> 
> 
> 
> 
> 
> 
> 
> Please note that the PCIe 3.0 is connected to both GPU and the CPU. Pre-C2 Xeon E5 CPUs were not validated at all @PCIe 3.0, which means - it might work, or it might not. Fortunately for people who bought retail Xeon E5, large majority of them are C2 (there was an initial batch of C1-stepping Xeons with SR0xx stepping), but as far as I know these only went to key OEMs and partners pre-launch, so chances of having one are slim. However, ES/QS ones are a mixed bag - not officially validated, and anything can happen.


Good to know. Here's to hoping I can get two running smoothly. Honestly there isn't a card out there that can fully saturate a PCIe 2.0 16x slot yet, let alone a PCIe 3.0. So I'm sure two Titans in PCIe 2.0 config would perform the same.


----------



## anubis1127

Quote:


> Originally Posted by *TheBlademaster01*
> 
> Nice, thanks man.
> 
> How do you change the bios IC, solder the 8 pins to the board?


It's removable, I just carefully grip it with some small pliers, pop it out, and put the new one in.


----------



## TheBlademaster01

Is it this chip?



Spoiler: Warning: Spoiler!


----------



## psyq3212

Yep.

That's the one.

It can also be hot-flashed if you have a spare


----------



## psyq3212

Quote:


> Originally Posted by *ViperJPB*
> 
> Good to know. Here's to hoping I can get two running smoothly. Honestly there isn't a card out there that can fully saturate a PCIe 2.0 16x slot yet, let alone a PCIe 3.0. So I'm sure two Titans in PCIe 2.0 config would perform the same.


Depends of the usage actually - I am using my TITANs for CUDA, so I'd appreciate extra bandwidth between the CPU and the GPU.

However, I have a Haswell setup for that


----------



## TheBlademaster01

Quote:


> Originally Posted by *psyq3212*
> 
> Yep.
> 
> That's the one.
> 
> It can also be hot-flashed if you have a spare


Hot flash?


----------



## psyq3212

Well, basically, on any ASUS board which uses the same chip (all of the ones I have do) you can first boot with the board's own chip, then remove it carefully while board is running, and then put the chip you want to "hot flash" in. While the new chip is in, perform the regular flashing process (e.g. with AMI's BIOS tools, or ASUS own tools) and once the flashing is complete, power down the board and return the original BIOS chip.

This only makes sense if you do not have a spare programmed BIOS chip and you want to bring up the board as soon as possible. It sounds dangerous but actually the most dangerous part is ensuring that the chip pins do not get damaged and no contact get shorted. Basically, you need steady hands and some patience.

I do not recommend this in general, but if someone's board is bricked due to CPU incompatibility, this is one of the ways to go.


----------



## ViperJPB

T
Quote:


> Originally Posted by *psyq3212*
> 
> Depends of the usage actually - I am using my TITANs for CUDA, so I'd appreciate extra bandwidth between the CPU and the GPU.
> 
> However, I have a Haswell setup for that


I would be using them primarily for CUDA. DaVinci Resolve and 3D rendering with CUDA. This a working mans workstation







. Currently using three GTX 680 4GB cards. However I have seen reviews of Titans in external expansion boxes, this running off a single shared slot not at 3.0, with little to no difference in CUDA performance versus the same two cards internal on seperate slots.


----------



## psyq3212

Cool - please let us know if you managed to get PCIe 3.0 x16 speeds with TITANs.

I now also have Z9PE-D16 at my home, so after I am back from a trip I will try to see if it is any better with regards to PCIe x16 3.0,


----------



## ViperJPB

Sounds good. So is Asus just spouting false advertising if these boards are having such difficult times getting to PCIe 3.0 speeds? Could it have something to do with the multiple Link settings that are disabled by default under the QPI settings?


----------



## ViperJPB

Quote:


> Originally Posted by *psyq3212*
> 
> Please note that the PCIe 3.0 is connected to both GPU and the CPU. Pre-C2 Xeon E5 CPUs were not validated at all @PCIe 3.0, which means - it might work, or it might not. Fortunately for people who bought retail Xeon E5, large majority of them are C2 (there was an initial batch of C1-stepping Xeons with SR0xx stepping), but as far as I know these only went to key OEMs and partners pre-launch, so chances of having one are slim. However, ES/QS ones are a mixed bag - not officially validated, and anything can happen.


How can you tell which version of the CPU you have? (C1 or C2)


----------



## psyq3212

HWINFO should be able to identify that.

If not, please tell me the SSPEC (QDF) code of your CPU (it should be SRxxx for retail / OEM, and Qxxx for ES / QS CPUs) and I will tell you which version it is.

As for ASUS and false advertising - well, this wouldn't be the first time. I still remember advertising that their 975X boards would support 45nm Core 2 CPUs... never happened.


----------



## ViperJPB

Didn't think to check CPUID or HWINFO. Have CPUID on the system already so I used it and it is indeed C2. So that's good. Just up to this board really then. And if indeed I can't get PCIe 3.0 ever to work I suppose I could always jump to another board someday. But I intend to get my moneys worth out of this one first.


----------



## BlackenedTush

There's been a lot of good discussion that I can relate to as always; but been too busy to contribute. But I did I just send a complaint to ASUS using their Official Complaint Page. I targeted 3 main issues:

the general finickyness of this mobo
the inadequacy of tier 1 tech support
the lack of technical information about the board in the manual or elsewhere
I suggest that ASUS send an engineer who is familiar with the board over to read this thread to get some insight as to the predominant questions/needs, and then have him/her directly contribute some posts containing relevant technical insight and/or dispelling any obvious myths. I also suggested that they might also contact registered users directly with surveys/queries, or that they might set up a few online Q&A seminars with engineers familiar with this board so registered owners or interested parties could attend and message in their questions, and finally, to write a technical addendum to the manual, especially explaining all things bios.

It wouldn't hurt to get more squeaky wheels in on the ASUS complaint train if you haven't already done so enough.

Cheers







BT,


----------



## lloyd mcclendon

**** BIOS UPDATE ** again* ... good things .. perhaps asus is picking up their slack
Quote:


> Version 5003
> Description BIOS 5003 for Z9PE-D8 WS released to public
> 1. Fix DEL key may not able to enter BIOS Setup.
> 2. Modify CPU2 temperature presentation while CPU2 not be installed.
> 3. Default "Enable" INTEL VT-d.
> 3. Fix memory size might different between POST and BIOS Setup.
> 
> [Warning]
> *If your BIOS version is 0703 or older: Go to BIOS-Utilities to download and flash the BIOS to version 3109 firstly before updating the BIOS.
> File Size 4,01 MBytesupdate 2013.06.19
> Download from Global


They got the numbering wrong







Should I trust these people to write a bios?









#3 (2) looks interesting, but I still don't think it's the silver bullet for all of our problems. Will try, but maybe next round

#1 maybe ... this particular USB keyboard does not work in the bios, it used to on the version it shipped with, then I updated iit and it stopped working


----------



## DizZz

I'm getting an "FF" post code which after looking online seems to be an error code yet my computer has been running perfectly fine and stable for the last week and a half. What should I be getting or what does it actually mean?


----------



## lloyd mcclendon

FF is code normal, that's what you want to see. I believe everyone has that when things are in good shape .. if not please correct me if I'm wrong.

PS since I see you're running linux .. help me help you. Can you try to suspend to ram and LMK if it comes back normally? sudo sh -c "sync && pm-suspend" I'm |__| this close to borrowing a drive and doing a windows install to see if even that will suspend...


----------



## DizZz

Alright thank you. I will check that tomorrow as I'm not currently home and I'll let you know!


----------



## ViperJPB

Hey all...quick update. PCIe 3.0 with all GPUs is attainable with this lovely executable from Nvidia.

http://nvidia.custhelp.com/app/answers/detail/a_id/3135/~/geforce-600-series-gen3-support-on-x79-platform

Works like a charm and I was able to get all 3 of my GTX 680 4GB Galaxy cards running at PCIe 3.0 16x, validated by GPU Z. Even with my Slot 1 GPU.

Now your mileage and benefits may vary depending on the video card and it's support for such. Mine worked...but I was getting better benchmark results in 3DMark and Cuda processing with PCIe 2.0, so I don't think my cards are ready for it. Granted I forgot to run the bandwidth tests in Sandra. Though others were seeing slight gaming performance improvements in PCIe 3.0. Seems to be the norm...zero to slight performance improvements. Will have to run this when I switch to Titans.

Thought everyone should know.


----------



## lloyd mcclendon

Well, the latest bios update again doesn't really do anything exciting. However ...

The "VGA card not detected at boot issue" .. I think is gone, at least I haven't seen it for a couple weeks. I increased the "Link training timeout" from 400ms (default) to 1000 (max) in advanced > pci > pci express settings. And so far the cards have always been present on boot.

Now what's really interesting, is after the bios update, that value is now magically set to 5000ms even though the max still says 1000.







So, that very may well have something to do with it, those of you struggling with that issue may want to look at that setting and/or update your bios.

The USB keyboard in bios still does not work, and I'm beginning to suspect the S3 resume is my linux kernel config or a problem module, or perhaps buggy ACPI tables that MS may work around (and now linux has the capability to workaround).


----------



## ViperJPB

Quote:


> Originally Posted by *lloyd mcclendon*
> 
> Well, the latest bios update again doesn't really do anything exciting. However ...
> 
> The "VGA card not detected at boot issue" .. I think is gone, at least I haven't seen it for a couple weeks. I increased the "Link training timeout" from 400ms (default) to 1000 (max) in advanced > pci > pci express settings. And so far the cards have always been present on boot.
> 
> Now what's really interesting, is after the bios update, that value is now magically set to 5000ms even though the max still says 1000.
> 
> 
> 
> 
> 
> 
> 
> So, that very may well have something to do with it, those of you struggling with that issue may want to look at that setting and/or update your bios.
> 
> The USB keyboard in bios still does not work, and I'm beginning to suspect the S3 resume is my linux kernel config or a problem module, or perhaps buggy ACPI tables that MS may work around (and now linux has the capability to workaround).


Interesting. Was curious about the new bios. I'm still wondering about those extra Link page settings myself.


----------



## LeMakisar

hi everyone !

They apparently removed the 5003 bios ??
I've been using it without a problem but now I'm wondering ...

just wanted to know if someone is playing games with this mobo. For some games it's pretty clear that the dual CPU isn't supported and there are tricks that works ... but I've been trying to make Race Driver GRID (the first one) work, without success .... would someone happen to have succeded running this game ?

Thanks !


----------



## lloyd mcclendon

They did remove it .. LOL. We're screwed. It seems fine though. I have the zip file here, if anyone wants to be adventurous PM me and I'll send it.









As far as gaming goes, I do, but it's through a 4 core VM so the real cpu topology isn't visible. No issues, it may just be a bug with that particular game.


----------



## LeMakisar

thanks for the info.
I'll try to use a VM and see what happens, I hope it will work.

Regarding the bios, it's odd they removed it if it worked fine ..... let's wait and see


----------



## psyq3212

Hmm, 5003 BIOS was not really working well for me - system sees only half of my RAM (I use 8 sticks of Samsung's ECC DDR3 16 GB RAM) and with some ridiculous timings.

I am going back to 3506.


----------



## BlackenedTush

Thanks Llloyd for the info


----------



## BlackenedTush

Thanx Viper... very interesting


----------



## psyq3212

By the way, for people who still want to experiment with this BIOS,

It can be found here:

http://www.pugetsystems.com/parts/Motherboard/Asus-Z9PE-D8-WS-8386

Direct DL: http://files.pugetsystems.com/files/2579/parts/Motherboard/Asus-Z9PE-D8-WS-8386/Z9PE-D8-WS-ASUS-5003.zip

However, like I said, this BIOS seems to be buggy - at least on my Z9PE D8 setup, so proceed carefully. In the worst case, you might need to revert back to 3506 like I did.


----------



## xorcs

Hi! 5003 firmware is terrible. Do not see a single CPU, and half the memory. I had to solder a spi programmer to roll back. Is it possible to force the board to work in version pcie gen2? gtx 770 is ignored, but the gtx470 works. sorry for my english.


----------



## phillydee

Great thread; I'm about to build a custom workstation for 3D animation, editing, and software rendering. A few questions:

I have a MSI 4GB GTX680 that I want to throw in here... is it going to be an issue? I'm also looking at some C0 stepping ES chips(Xeon E5-2665 or 2680 depending on what's available). Will I encounter problems with the current working BIOS (3506?) moving forward?

Finally, what's a good EEB case? I'd like it to look modest at best... thanks!


----------



## psyq3212

C0 steppings are OK, as they are qualification samples and functionally equivalent to the commercially available SKUs.

Problems happen with pre-QS samples (in this case ES2 generation - B0 and B1 stepping) as support for them was probably dropped long time ago in the BIOS/UEFI reference code, and I would be quite sure that nobody in the motherboard vendor company would keep testing pre-QS silicon with BIOS updates. Interesting enough, B0 still works with Z9PE D8 WS.

The only problem with C0 is VT-d support that had to be disabled due to a bug, which was solved in later steppings. Unless you heavily use virtual machines this should not concern you too much - but if you do, you should get C2 stepping which is identical to the shipping sillicon.


----------



## derickwm

Silverstone has some nice EEB cases. TJ07 is a classic, currently looking at getting a GD07 myself if I end up getting another one of these boards for IB-EP 

http://www.silverstonetek.com/product_case.php?go=Y&area=en

Just sort by EEB to see available ones.


----------



## phillydee

Quote:


> Originally Posted by *psyq3212*
> 
> C0 steppings are OK, as they are qualification samples and functionally equivalent to the commercially available SKUs.
> 
> Problems happen with pre-QS samples (in this case ES2 generation - B0 and B1 stepping) as support for them was probably dropped long time ago in the BIOS/UEFI reference code, and I would be quite sure that nobody in the motherboard vendor company would keep testing pre-QS silicon with BIOS updates. Interesting enough, B0 still works with Z9PE D8 WS.
> 
> The only problem with C0 is VT-d support that had to be disabled due to a bug, which was solved in later steppings. Unless you heavily use virtual machines this should not concern you too much - but if you do, you should get C2 stepping which is identical to the shipping sillicon.


Hey thanks for the quick reply. I'm definitely not doing any server/VM type use and don't plan on using VMWare etc once I'm up and running, so this is good news.


----------



## phillydee

Quote:


> Originally Posted by *derickwm*
> 
> Silverstone has some nice EEB cases. TJ07 is a classic, currently looking at getting a GD07 myself if I end up getting another one of these boards for IB-EP
> 
> 
> 
> 
> 
> 
> 
> 
> 
> http://www.silverstonetek.com/product_case.php?go=Y&area=en
> Just sort by EEB to see available ones.


I'm concerned about picking up a discontinued model; at least I thought most TJ series cases from Silverstone was discontinued....? Will need to check that--thanks!


----------



## anubis1127

I definitely like the GD07B as well. I'm looking at picking one up for my z9 board. Just haven't made a final decision.

phillydee, Here's a selection at newegg. Most of them are 4U server type cases though, which I'm also considering for my build, but doesn't necessarily sound like something you want, maybe.

For a more traditional upright PC case, the RAVEN RV03, or 04 would work, those are a bit odd for my taste. Cosmos II would also work.


----------



## DizZz

Quote:


> Originally Posted by *anubis1127*
> 
> For a more traditional upright PC case, the RAVEN RV03, or 04 would work, those are a bit odd for my taste. Cosmos II would also work.


I have mine in a raven rv03 but you can also fit it into a switch 810 or a haf x.


----------



## TheBlademaster01

My ultimate SSI EEB case would be a Lian Li V750, nice and compact









Just to make sure, do B0s work with .cap BIOSes? Also do they miss VT-d as well (not that I'm going to use it though)?


----------



## phillydee

wow the QVL is tiny... I'm looking at a set (or two) of these:

http://www.newegg.com/Product/Product.aspx?Item=N82E16820239234

Really don't want anything wild at this point, just want this thing to run well is all. I'd like to get 64GB in the machine, so I was thinking a quad channel set x 2.

Thanks in adv.


----------



## ikem

Quote:


> Originally Posted by *TheBlademaster01*
> 
> My ultimate SSI EEB case would be a Lian Li V750, nice and compact
> 
> 
> 
> 
> 
> 
> 
> 
> 
> Just to make sure, do B0s work with .cap BIOSes? Also do they miss VT-d as well (not that I'm going to use it though)?


i would like that too but im still content with my v1200


----------



## TheBlademaster01

I would be content with a v1200 as well. Have to settle for my RV01 though


----------



## anubis1127

To answer your question regarding VT-d, no, it's not supported. It's actually not on any of the ES chips I've come across.

Quote:


> Originally Posted by *phillydee*
> 
> wow the QVL is tiny... I'm looking at a set (or two) of these:
> 
> http://www.newegg.com/Product/Product.aspx?Item=N82E16820239234
> 
> Really don't want anything wild at this point, just want this thing to run well is all. I'd like to get 64GB in the machine, so I was thinking a quad channel set x 2.
> 
> Thanks in adv.


I would honestly take the QVL with a grain of salt. I bought a kit off it to specifically run it at the XMP rated speed, and I've had no luck on this board. To get 64GB you could still use unregistered DDR3 and save a little money. If you want more than 64GB on this board you have to go with the registered DDR3.


----------



## phillydee

Quote:


> Originally Posted by *anubis1127*
> 
> I would honestly take the QVL with a grain of salt. I bought a kit off it to specifically run it at the XMP rated speed, and I've had no luck on this board.


Wow OK thanks, that makes me feel a bit safer about my choice. I'll glean over RAM again then since I don't think I'll go over 64GB with this build.


----------



## psyq3212

Looks like ASUS released version 5004 online.


----------



## xorcs

Who used the new version of bios? The problem gtx680 & gen3 Fix?


----------



## LeMakisar

I'm currently with 5003 but I'm considering updating to this version, I'll keep you informed

edit : done







I'll have to test it, but as it was said before, with 5003 my ram timings were quite funny, everything seems to be in place with this 5004.


----------



## brock2621

Ok so I just ran across this thread and it's making me nervous about this mobo. I plan on putting 2x 2687w's with 64gb Corsair Vengeance 1600 DDR3 Quad Channel, and an asus GTX Titan and now I'm concerned about compatibility and quality.

Should I pick up the discontinued sr-x instead? Or maybe a supermicro? I just liked the native USB 3 of the board.

All suggestions would be welcome









P.s. this is mainly an AE and C4d board


----------



## LeMakisar

all I can say is I have dual 2687W (but not es), 64GB G.Sklill PC2133, and a titan, and absolutely no issue.

Though my first Z9PE litteraly burnt one of my 8pin CPU power cable .................

To be honest, there isn't really THE dual s2011 mobo, that's why I took the Z9PE which is fine (although I miss the EVGA warranty extensions ...)


----------



## brock2621

Quote:


> Originally Posted by *LeMakisar*
> 
> all I can say is I have dual 2687W (but not es), 64GB G.Sklill PC2133, and a titan, and absolutely no issue.
> 
> Though my first Z9PE litteraly burnt one of my 8pin CPU power cable .................
> 
> To be honest, there isn't really THE dual s2011 mobo, that's why I took the Z9PE which is fine (although I miss the EVGA warranty extensions ...)


Curious on the ram choice? Isn't it limited to 1600?

You mean engineering sample by "es?"

I think I'm ordering all my stuff this week so ill keep you updated. What bios version are you running?


----------



## squidman

Need help with an ASUS Z9 PE-D8 WS build for 3d rendering (both CPU as in Vray/Vue XStream and gpgpu as in Octane) as well as video editing, stabilizing, compositing in nuke/aftereffects etc.

New to the forum, but have read MOST of the 32 pages in the thread as well as countless others. Based in Norway, so have somewhat limited hardware choices (especially regarding budget!). Ditto regarding cases.

Proposed build:

2x Xeon E5-2620...pretty much all I can afford! Still a grand US here (25% VAT/GST! + markup!)

Crucial DDR3 Ballistix Sport VLP 32GB Kit KIT (4x8GB) 1600MHz CL9 UDIMM 240pin (PC3-12800) Very Low Profile

XFX ProSeries Black Edition 1250W PSU: ATX 12V V2.31, 80 Plus Gold, Modular, 8x 6+2pin PCIe, 11x SATA, 8x Molex

El-cheapo Samsung SSD 840 Series 250GB 2.5" OEM: Basic Kit, MDX controller, 540/250MB/s read/write for caching

Seagate Barracuda® 2TB SATA 6Gb/s (SATA 3.0), 64MB Cache, 7200RPM, 3.5" - to be used with smart-cache above cheap SSD

Cheap graphics card for now...can't afford titan! EVGA GeForce GTX 760 4GB PhysX CUDA..need the VRAM for Octane....can always add more.

Maybe a cheap firepro in addition for Max/Maya etc..e.g. AMD FirePro V4900 1GB GDDR5 to drive the display....

Case: almost no SSI-EEB cases here except for Cosmos II (which I think is ugly and fits little for being such a huge case). Can get Corsair D900 for same price and ghetto-mod it, or any other EATX case (not averse to drilling!). Would prefer to save a bit here (say spend 180 USD instead of 500 USD for the above!). Would Fractal designs R2 LX work? Any other case suggestions? Leaning towards very big case/HPTX in case I start sticking lots of graphics cards for GPGPU rendering. Or maybe I just do a shopping at Ikea!

Cooling: leaning towards Noctua air-cooled as Xeons don't overclock, but may go closed-loop (e.g. 2x Corsair H70 Cooler + 2x SP120 Quiet Edt.) if I can them to fit. Bit concerned about stiff Corsair tubes/pipes. Or a custom watercooling setup, but expensive and would be first time I try it.

The other option is a 3930k build which I overclock to within an inch of dying, would be at least a grand cheaper, and build rendering nodes using cheapest amd setup possible (amd 8150 or similar). Problem is the cheapest I configured a render node that way was around 4500 nok or 750 USD, so not really saving anything. And I would lose out on the 7 pci-e slots.

Thanks, sorry for long winded post! Let me know if you see any incompatibilities, ASUS site with approved parts is hopeless. And long list of complaints/RMA's on Newegg regarding board is scary...but still lust after it!


----------



## TheBlademaster01

I'm pretty sure that a 4.5GHz+ 3930K will be faster than dual E5 2620 though also a lot more power hungry.

RV03 is the cheapest SSI EEB case that comes to mind.

I bought a used Z9PE to make sure it was working. My perception of this board is that it's either DoA or works properly for the rest (with some features broken perhaps like PCI-e 3.0).


----------



## squidman

Thanks I read you there! Another alternative (but quite a bit more expensive!) is too stick with just one 8-thread low-end xeon, and save up for another later (hoping they come down in price, not likely). The 1500-3k xeons are just out of my reach, and for that money I can build an AMD render farm a la Helmer. Thing is, I don't think this board works too too well with one processor...and at the moment I'm working on a laptop (i7-2630qm) which is 2-3 years old, so even 2x 2620's will be a huge improvement. Yea the RMA record and DOA record of this board isn't too cool....! This is not for mission critical work, I freelance, but wow it would be great to save on render times, (especially VRAY and mental ray and Arnold) ditto Vue Xstream, although apparently this board is a bit of a nightmare with Octane render and the 670-680-690 gpu's (GK104/keplar). Haven't found any posts on Titan/GK110/GTX 780 yet. It's also getting a bit long in the tooth.....

However, for CPU intensive rendering, don't have many alternatives here....don't see an equivalent kick-ass board from supermicro (and in any case would have to import from the US, with 25% tax on top of initial purchase and shipping).

The render node solution using AMD is also power hungry, and AMD even the 8350 is pretty friggin' slow...and like I said, it means setting up distributed rendering, and spending minimum 4500 NOK or 750 USD (per node) whereas the e5-2620 retails here for 3000 NOK (or 495 USD). Of course I save around 750-1k USD setting up an equivalent 3930k system as opposed to dual xeon system.

Horses for courses! Maybe splash some cash on dual xeon system with one better specced 8-core? and hope by next year the whole system isn't completely obsolete?


----------



## squidman

Addendum,

Well, I get that this board is nearing obsolescence and has many RMA and DOA problems, and I can't afford 2x top-shelf 8core xeons, but I figure....7 pci slots, and 2 lga2011 slots should give me a lot of upgrading potential. (iff I survive grey hairs and apparently horrible asus support!)..and an equivalent enthusiast single 2011 board is only 100 bucks cheaper. So iff it's too slow, I can upgrade processors (unlikely, by that time there will be a new form factor in say 2-3 years), or whack in lots of cheaper video cards with lots of vram for gpgpu (c'mon, when is opencl rendering going to take off! look at the 7970), and worst case it's relegated to my growing museum of computers for use in render farm. I could probably still game on it too, although I only have the one 1920x1200 monitor at the moment (foresee a 4k monitor in say 1-2 years time).

Still, buying when expensive, and paying top dollar in this game, you're always going to lose unless it makes you money back very quickly. They're obsolete by the time they're released.


----------



## TheBlademaster01

Well Ivy-EP is coming with 12 cores and higher IPC, so







...

Could be here in September or October.


----------



## squidman

Will that be a new form-factor/socket? Please say no! Still waiting for developers to optimize code for xeon phi (now THAT would be a rendering beast!)...

Edit: I badly need a workstation within the next month....anyone care to comment whether my build is going to work at all or not? Or whether it's feasible to whack in one CPU and one GPU (still Keplar) and have the thing boot? Ditto choice of PSU? Unable to send a simple message to ASUS support without having actually purchased the damn thing (wanted to ask about compatibility of parts), doesn't promise well...maybe supermicro in my build after all...


----------



## TheBlademaster01

It should be supported by this board with a BIOS update. There will be no new socket or chipset. Haswell needs a new socket though.


----------



## squidman

Haswell is the ultra low TDP chip, so don't see how that applies here. Gotta bail (trying to recover gopro files!). Thanks for the responses.


----------



## psyq3212

I am going to get two Ivy Bridge EP Xeons (highest core count) in the next 2-3 days, and I will test them on both Z9PE-D8 WS and Z9PE D16.

As far as I can see, Z9PE D16 has recently got a BIOS update containing latest MRC version, so I suppose Z9PE D16 should work - but I will test first and then confirm.

As for the Z9PE D8 WS, ASUS is less transparent on the MRC version they use in BIOS-es for this board so I am not sure if it already works or not, but I will test as well.

In any case, both boards should definitely be compatible with Ivy EP once they get proper MRC and microcode update.


----------



## TheBlademaster01

Quote:


> Originally Posted by *psyq3212*
> 
> I am going to get two Ivy Bridge EP Xeons (highest core count) in the next 2-3 days, and I will test them on both Z9PE-D8 WS and Z9PE D16.
> 
> As far as I can see, Z9PE D16 has recently got a BIOS update containing latest MRC version, so I suppose Z9PE D16 should work - but I will test first and then confirm.
> 
> As for the Z9PE D8 WS, ASUS is less transparent on the MRC version they use in BIOS-es for this board so I am not sure if it already works or not, but I will test as well.
> 
> In any case, both boards should definitely be compatible with Ivy EP once they get proper MRC and microcode update.


Where did you get them







?

I wish I could get some lol


----------



## LeMakisar

Quote:


> Originally Posted by *brock2621*
> 
> Curious on the ram choice? Isn't it limited to 1600?
> 
> You mean engineering sample by "es?"
> 
> I think I'm ordering all my stuff this week so ill keep you updated. What bios version are you running?


I already had 1 set of this RAM, and in fact I had a pretty good deal for the second one









Yes, ES is for engineering sample

I'm running on bios 5004


----------



## BlackenedTush

brock2621 & squidman,

There's stuff for both of you here... so pick and choose.
And squidman, you think that YOU are longwinded...









*About Cases*...
I had trouble finding an EEB case myself. I didn't understand the differences between EEB and the same-sized EATX form factors.
So I looked at the pricey EEB Cooler Master Cosmos II. It was HUGE enough for an easy build; but it was too big to fit under my desk. And there is a lot of "aerodynamic laundry hanging out to dry" on the outside (making it look kinda like a piece of exercise equipment). I could have gotten used to that but going to a more mid-range case I knew I could save enough money to buy another SSD. I almost bought the EEB compatible NZXT Switch 810; but it was not quite as sturdy as the EATX compatible Cooler Master HalfX. I settled on the HalfX and I'm very happy with it. It's got great airflow potential, it's well built, has two 3.5 hotswap bays for BU rotations off-site, plus it's roomy enough for my RAID array (five 3.5 drives), four 2.5 drives, and three optical bay devices. Again, the looks are more gimmicky than I would have liked; but it's a workhorse.

But with this choice of case came the EATX issue. I found that 2 out of the 10 pre-drilled standoff holes didn't line up with the board holes. One option I've seen people suggest is to leave them off. I wouldn't recommend that with this board, as both of them are found under vast spans in the middle where there's very little surrounding support from other standoffs. One is actually in the proximity of the RAM slots so the mobo would definitely flex a lot if you try to replace RAM after the board is installed. You could drill and tap new holes in the case; but that comes with it's obvious difficulties. My approach was a compromise. I filed the threads off of the brass standoffs which came with the case. Since I had a vice to hold them securely during the process, it was fairly easy. The boards little mounting screws still attach them to the board; but they only rest on the case surface. But support is very decent.

Gaming Pedigree SSI-EEB cases:
Cooler Master Cosmos II (~$330 US)
NZXT Switch 810 (~150 US)
Silverstone Raven RV01 (~$240 US)
Silverstone Raven RV03 (~$150 US)
Silverstone Raven RV04
Chenbro CA-10569BK (~$90 US)

Gaming Pedigree EATX cases:
Mountain Mods custom (~$250-$600 US)
Corsair Obsidian 900D (~$350 US, very big & simple; but latest model with tons of drive space & flexibility)
Corsair Obsidian 800D (~$280 US)
Corsair Obsidian 650D (~$190 US, note: 3-screw, no mod install of Z9PE-D8 - http://forum.corsair.com/v2/showthread.php?p=578030)
Cooler Master HALFX (~$190 US)
Rosewill Blackhawk-Ultra (~$180 US)

Server Pedigree SSI-EEB cases (most with PFC PSUs):
iStarUSA D-490S rack-mount series (~$240 US)
Intel SC5600LX (~$900 US)
Intel SC5650WS (~$400 US)
Intel SC5600BRPNA (~$730 US)
Intel SC5400LXINA (~$680 US)
Intel Server Chassis P4308XXMFEN (~$400 US)

Media-Center/HTPC Pedigree SSI-EEB cases:
Silverstone Series SST-GD07B (~$140 US)

*
My experience with Compositng software on this mobo.*..
I haven't a lot of experience with Nuke yet; but I've done some work with the PE version and I've done the 2-week temporary tryout of NukeX. In NukeX I did some 3D tracking on 480 frames worth of 1920 x 1280 HD footage. I haven't detected any performance issues. It's been stable, seems snappy enough, and rendering is quick. But I don't have much to judge it against.

I have had really good results with AE (CS5 & CS6). When building my machine, I paid attention to the Adobe CS5 performance white papers. So I got the largest number of cores I could afford (E5-2630s) and chose 64GB of RAM. I also came up with a multiple-drive strategy as well: SSD boot drive, SSD application cache drive, and a RAID 10 HDD array as storage/rendering target drive... (plus BU HDDs). With the 12 physical cores and 64GB of RAM, I am able to go into preferences and assign to AE a total of 19 virtual cores and the maximum allowable RAM for each core (3GB) for fastest rendering performance. That still reserves 5 out of my 24 virtual cores plus 7GB of RAM for the OS and other non Adobe apps. Adobe says that there's a possibility of over assigning cores; but I haven't seen proof of that yet. And they don't explain what sort of composite will exhibit this behavior. Recently I did a series of tests in CS6 with a fairly complicated composite. But one experiment like this couln't possibly prove or disprove their point. Anyway, it did serve to show me the benefit of more cores. It contained several varieties media: two different codecs of video, sequences of large PNG files with alpha, multi-layered Illustrator and Photoshop files, effects like Blur, several effects from the Trapcode Suite, text with animated effects, and more. I started with 4 cores & the maximum RAM per core (3GB). I worked my way up to my max of 19 virtual cores with 3GB/core. Performance improved in linear fashion with the addition of each core. What took 12 minutes with that max setting, took over an hour with "Render Multiple Frames Simultaneously" turned off. And when I set it to render with the new Ray-Traced 3D Renderer (using my GTX670 GPU), it took 2 1/2 hours! But that's because I didn't have any 3D in the composite. I have yet to test the pure GPU Ray Traced 3D Render with that type of composite. But I really love how Adobe and Octane are focusing on exploiting the procesing power of Nvidia CUDA technology. That helps digital artists on a budget because they can use gaming cheaper/faster gaming cards. Neither uses SLI; but that doesn't matter. If you have multiple cards they will use all of the CUDA cores that you have installed. But AE does limit the total RAM used for GPU processing based on the card that has the least amount. So if you install a 4GB card and a 2GB card, it will set the maximum for each to 2GB, for a total of 4GB.

*My experience with 3D modeling software on this mobo*...
I am not that well versed at C4D yet; but I have played with it and it is quick and stable. Autodesk Maya seems stable on my machine as well. I do have issues occasionally with 3DsMax tho. But who doesn't? 3DsMax is a buggy patchwork of code. On my machine I have seen strange geometries fly out from my quad's verticies on screen during rotation of complicated geometries. Also it crashes when I try to do _Camera Per Pixel Mapping_. But I have seen this behavior with this program on several other different HP workstations (didn't check the GPUs). The screen issues make me wonder if the Quadro/Tesla/Maximus approach that Autodesk recommends is more than just words. Like many peeps these days I am using a gaming GPU because price/performance is superior to workstation GPUs... And Nvidia GeForce particularly for Adobe applications. My card is an EVGA GeForce GTX670 Superclocked w/ 4GB. The NVIDIA gaming cards don't get the benefit of worstation level drivers tho, so the precision is reduced. Plus there are other processing technologies within the Quadros that are vital to 3DsMax that are disabled in the GeForce cards. I know that there are some ways of hacking various GTX models to turning them into faster versions of their Quadro counterparts; but where one's livelihood is at issue, the stability & reliability found in stock components is more important.

*About the SRX as an alternative mobo*...
The EVGA SRX wasn't available when I got the Z9PE-D8 over a year ago; but even so, it looked pretty cool and I was drooling over it. Since then I haven't seen too many reviews about the SRX. I just happened upon one the other day by Tom Logan of Overcloc3D, so I watched it eagerly. He is as usual long winded, and he didn't have much good to say about it. He whines a lot about color mismatches (which is probably silly to anyone building a WS); but he's more into the OC/gaming culture where that is part of the game. He does make some good points tho about the SRX's practical/technological inadequacies. He thinks the ASUS board is much superior.

EVGA SRX mobo review:
http://www.overclock3d.net/reviews/cpu_mainboard/evga_sr-x_classified_review/11

ASUS z9PE-D8 mobo review:
http://www.overclock3d.net/reviews/cpu_mainboard/asus_z9_pe-d8_ws_dual_xeon_insanity_e5-2660_e5-2687w/12
*
GPU/Display For Octane*...
I don't believe it would be money well spent getting an additional GPU just to drive the display, if that's what you are saying. The GeForce cards would support your monitor(s) using a bit of RAM; but as with Adobe Rendering apps the more CUDA cores you can throw at a render the faster it will go. So if you double up, go with two identical Kepler Nvidia cards. It's easy to go overboard with complex variations and it easily causes unseen issues. If two of the same don't work, send them back and try two others.
Here's a thought... If the board isn't too touchy with 3GB GTX 660s or a Titan (I don't know the latest on compatibility and the BIOS situation), two 660s would supply the same number of CUDA cores & RAM to Octane, AE & Premiere as a Titan GPU costing double that. That's 2688 CUDA cores and 6GB of RAM. There may be single cards or dual setups that will hit your value/performance sweet spot better.

*Smart Caching*
I haven't taken advantage of the smart caching feature of this board. I simply used an SSD as a boot drive. But i had issues that caching might have circumvented. Because my boot drive is only 120GB, I had to manually optimize my setup to make efficient use of the limited space available. First, I had to disable sleep/hibernation feature because Windows 7 reserves C: drive space for state storage. Whatever your RAM Capacity is, that's how much space Windows reserves on the C: drive. Because I have 64GB RAM, over half my drive would be taken up for hibernation purposes. There would not be enough room for all of my applications. I also needed to move all of my personal folders from the C: drive to my storage drive (which I'd probably do anyway). I disabled search indexing., disabled the pagefile on my c: drive, and enabled write caching. Using board driven caching would conceivably give you similar boot performance; but without the setup backflips I just described.

*Water Cooling*...
These Xeons don't get very hot at all. My wimpy Dynatron R17s in my very well-ventilated case, plus dry 78F/25C ambient temps keeps CPU1 averaging around 38C and CPU2 at around 42C. I'm sure that the back one is hotter because it sits in the warmer air stream of the first. Using the H70 water cooling like you suggested would conceivably even out their temps a little bit and also drop temps down a few more notches; but it isn't necessary. And a custom rad that they both shared would definitely even them out. I didn't care for the stiff hoses on that cooler and didn't like the idea of potential leaks.

*Dual Xeons vs OC'd i7*...
Not much of a contest if your board works and you can afford it. The Xeons with their superior number of cores will be all around workhorses that will outperform the single CPU in the widest range of 3D and rendering situations. It is possible that a fast, single multi-core CPU like an OC'd i7 could edge out a pair of slower E5-2620 Xeons in rendering if you're using Octane and a couple of fast CUDA GPUs. It's hard to say tho without testing. And I'm sure that in general Windows performance, the OC'd single CPU would feel snappier with the fast i7 than with a pair of Xeons... no matter what they were. But you'd need to weigh those 2 situations in your workflow against what the better general grunt of more cores would buy you.

*XFX ProSeries PSU*...
I don't know how good this PSU is but at first glance it does look decent. It employs active PFC which is good. Since it does, you will need to get a PFC compatible UPS if you are choosing to add one to your collection. And I'd recommend that you do if you live in storm country or have flaky power. "Active PFC compatible" UPS units are typically of the "Pure Sinewave" type and unfortunately more expensive. But without a compatible one, and in the event of a power failure, your PSU will very likely shutoff during the short transition to battery backup power. So what's the use in going to the trouble and expense of setting up a UPS if that happens? I have the CyberPower 1500PFCLED. It has saved my butt several times while I was working to stringent deadlines.

This is all my opinion of course but it's backed by blood, sweat, and tears. Your mileage may vary..

Go getem.

BT


----------



## ikem

There are a bunch of Lian Li's that are EEB compatible.


----------



## TheBlademaster01

I meant 3930K at 4.5 or above vs 2x 2620s. The 3930K will win.


----------



## beaker7

Quote:


> Originally Posted by *ikem*
> 
> There are a bunch of Lian Li's that are EEB compatible.


+1. Currently running a Z9PE-D8 + E5-2687w's in a PC-90 and its great. Going to rebuild to an A75 once the new chips come out for a bit more drive space though.


----------



## Jann

Xigmatek Elysium works ok with this motherboard. I don't like the hdd cage, but everything else is ok. And it was cheaper than anything else in that size. A friend has the similar setup in a Haf X, but it's very tight.

For performance, I can say that a single 3930k OC'd to 4.6-4.8 comes very close to two E5-2660 8 core xeons. So in most cases i7 is a better buy. Even more so if comparing two 6 core xeons vs 3930k.
As for board problems I've had none, except maybe that my supposedly 1600 ram only works as 1333 on auto mode. If I force it to 1600, 3 out of 8 chips show up as 4Gb instead of 8. But I think that's Ram's problem, not the board.
For cooling I have two simple CM 212 Evo's, and they work very fine. Around 60c max temps when on full load even now (summer). 35-40 on idle. So I don't see a point in watercooling anything less than E5-2687W


----------



## lloyd mcclendon

Yeah you really don't need to watercool anything .... mine are around 28C idle / 36C load .. and probably would be better if I redo the thermal grease properly this time







I had the whole WC thing going before I went to this platform, so I just kept it, but really am debating just simplifying and going back to air one of these days.

As far as the board goes, I would say it depends on how adventurous you are. When it works, it's an amazing platform. When it doesn't, bring your patience / frustration management hats. It's sort of like owning a heavily modified car if any of you are familiar with that. Awesome project, at times a good thrill, but day to day "something's always breaking" and/or it's a money pit. However eventually I figure everything will stabilize. Right? It's at least fun and exciting, something to do, a good hobby. Right?

It certainly looks like they are still actively working on the BIOS, lets hope this continues until the point where it is no longer needed. I think they still have some work to do.

----
In other news, the PCI card not detected problem does indeed seem to be fixed with increasing the "link training timeout" and update to 5004 as I mentioned in my previous post. It hasn't shown up at all in over a month, whereas before it would happen every 3 or 4 days. So I've got that going for me, which is nice.

I'm also getting closer to getting suspend/resume working. I actually discovered the wakeup hang was apparently due to a software problem on my end, for some reason stopping my raid 10 (linux software raid) before sleeping allows the PC to wake up. I suspect there's a bug in the raid kernel module, but have a fine workaround for the time being

However ... now this is where it gets really weird. If I let the PC sleep for 10 minutes, everything wakes up fine. If I let it sleep for an hour or more, upon hitting the power button to wakeup, the fans turn on, but the power led continues to flash and the motherboard never powers on. Strange. In reading around about this issue, it appears that it is actually due to a slight incompatibility between certain ASUS boards and certain CORSAIR power supplies. Either the ATX v2.x spec is slightly off or incompatible on either side, and/or the 5V charge from the unit is not within the boards tolerance on power on.

Now that seems like a long shot but it fits within 20 or 30 others with the same problem on a few different ASUS boards. I have purchased a different ANTEC 1000W unit to possibly replace my AX1200, we'll see if that fixes it later tonight. Stay tuned.

I put in a call to ASUS tech support about this. Explained my whole problem to the guy, as expected he rambled on about "windows update" ... "windows drivers" all sorts of nonsense. I explained 4 times it was linux, he put me on hold for 30 minutes, and then HUNG UP on me. Sent an email "sorry we were disconnected, I found a solution, go to the network control panel and enable something something."
















I called back and spoke with a guy who at least understood but didn't have any good idea. I tried to escalate through to an engineer, "is there anyway I / we could get an engineers time to answer some questions about this board" and he simply said "I can send an email to Taiwan but I probably won't get any response."







We really need to find a way to get one of those guys in this thread...


----------



## BlackenedTush

lloyd,

Thanks for the update on the PCI thing.

Good to hear you have some _good_ news with your board... you have that going for you... which is nice.

Sounds like your water cooling keeps your hotter (E5-2687w?) CPUs at a whopping 10 degrees cooler than my wimpy E5-2630s with Dynatrons; but also, it keeps their respective temps even. So you have that going for you... which is nice.

And about the sleep thing... As I said earlier, I have the same PSU. It's conceivable that I beez escaping your issues only because I have sleep/hibernation disabled. Wish I could verify for us. But it's amazing how the subconscious never sleeps. Doggedly... persistently, striving until our issues are solved! We often need that with this mobo. I think that ASUS should pass on a massive rebate to any such "Tier2 customers" like you for instance who work tirelessly to solve all of their deeper issues for them! I mean hey... They give free boards to reviewers. Just sayin'. I think they must have taken a page out of Microsoft's book, "Getting Rich by Launching With Beta Products and Letting Power Users Isolate the _Undocumented Features_".

I think your recent call to support reveals one of their biggest failure modes in responding to user issues adequately.
IMHO it's _language_ - Tier1/2 support speaks only broken English, the engineers familiar with this board's design can't look over our forum, or get involved in answering our questions because they speak only Chinese, and phone disconnects are merely a cover. So we have that going for us... which... Nevermind.

BT


----------



## BlackenedTush

Jann,

Yeah, my Half X is tight; but I really like it.

What you say about the i7 compared to the e5s is interesting. I would have predicted that those results would happen in certain instances. I'd really like to hear more. What applications did you test them on? Were they multi-threaded, 3D modeling, Color Correction, Video, preview or rendering, etc., ....?

I had similar RAM problems. It turned out to be bad RAM (just as you suspect with yours). I wonder how many builder issues are attributable to ESD... I'm not one to wear a wrist strap - during this build from scratch, or countless upgrades to other computers. But I'm always careful to manage static and maintain frequent or continuous contact with my case and mobo when installing.

You reported good temps with your _CM 212 Evo_'s (which I just purchased myself for another build, BTW).
But I wonder what your ambient temps are contributing to keep those higher-clocked Xeons so cool. Looking at your profile pic, I get the vision that you are sitting there in beanie, gloves, boots and parka; with the front door open; and a crisp, snowy wind blowing in.







That's cheating you know... haha

BT


----------



## brock2621

Ok so I'm in the need for some input. I'm about to build a new workstation for my job and am wanting to make sure I won't run into any hitches. I'm really wanting the best and fastest workstation available for After Effects work.

Silverstone tj10
PCM power and cooling silencer mk III
Z9pe-d8 (obviously)
Dual Xeon 2687w's
64gb's of crucial ballistix quad channel 8-8-8-24
Dual nh-d14 court heat sinks (is this possible with this particular MoBo and case?)
Asus GTX Titan
2 Samsung 840 pro 512 SSD's

How does all this sounds and look to you all? All input welcome and very appreciated!


----------



## phillydee

Progress update:

Thanks for everyone who chimed in with comments and thoughts. Build is going pretty well. Today's objective was to just get to the BIOS and make sure the CPUs and RAM were good. Aside from not starting as soon as I'd have liked, overall I'm pretty excited at this point.

A few thoughts:
- the C0 stepping ES models I ordered work flawlessly. (E5-2665)
- BIOS out of the box recognized both CPUs at the rated 2.4ghz.
- Using a temporary Gforce 9800 cheapo card in PCI lane 3.
- The Unbuffered server RAM (4x8GB) is running at 1600 without any BIOS tweaking
- I got a "Chassis Intrusion" message the first time I fired the system up. Weird? Hasn't happened since. I also don't have any of the case wires plugged into the mobo yet.
- The Coolermaster Cosmos II *DOES NOT* have all the proper mounting holes, despite it's claims that it fits EEB. I guess the EEB formfactor varies greatly in positioning of mounts... if I had a re-do I'd probably get a different case.

[edit]

Just noticed that if I change anything in the BIOS currently, (like, turning off the Asus logo) after warm reboot both CPUs read 1.2ghz in the BIOS... I'm hoping a BIOS update fixes this.


----------



## Jann

Quote:


> Originally Posted by *BlackenedTush*
> 
> What you say about the i7 compared to the e5s is interesting. I would have predicted that those results would happen in certain instances. I'd really like to hear more. What applications did you test them on? Were they multi-threaded, 3D modeling, Color Correction, Video, preview or rendering, etc., ....?


First I checked synthetic tests like Cinebench. And then I found a Russian blogger who compiled real world Vray testing from many users using three scenes of various difficulty.
You can simply look at the tables provided *here*
The green colored Farm test is the heaviest and that's where xeons shine.
A 3930k @4.6 got 21 mins, and my two E5-2660 got a bit under 15 mins, so that's why I think even @4.6 the 3930k is a better pick for price/performance, and if you get it to 4.8 then you have a winner. (and there's a WS mobo for single 2011 socket with 7 pci-ex if that's what you need)

All this is purely from a 3D rendering perspective. I haven't done any AE or other video production related testing. (if there are any good real life tests out there, let me know and I will)
Quote:


> Originally Posted by *BlackenedTush*
> 
> You reported good temps with your _CM 212 Evo_'s (which I just purchased myself for another build, BTW).
> But I wonder what your ambient temps are contributing to keep those higher-clocked Xeons so cool. Looking at your profile pic, I get the vision that you are sitting there in beanie, gloves, boots and parka; with the front door open; and a crisp, snowy wind blowing in.
> 
> 
> 
> 
> 
> 
> 
> That's cheating you know... haha


Currently both outside and inside office it's around 25c. It gets cooler only during winter, but then it's mostly the same temps due to heating


----------



## lloyd mcclendon

Quote:


> Originally Posted by *brock2621*
> 
> Ok so I'm in the need for some input. I'm about to build a new workstation for my job and am wanting to make sure I won't run into any hitches. I'm really wanting the best and fastest workstation available for After Effects work.
> 
> Silverstone tj10
> PCM power and cooling silencer mk III
> Z9pe-d8 (obviously)
> Dual Xeon 2687w's
> 64gb's of crucial ballistix quad channel 8-8-8-24
> Dual nh-d14 court heat sinks (is this possible with this particular MoBo and case?)
> Asus GTX Titan
> 2 Samsung 840 pro 512 SSD's
> 
> How does all this sounds and look to you all? All input welcome and very appreciated!


It looks very expensive. Hard to justify the cost of those chips isn't it?







It looks fine to me except I cannot comment on the graphics card. If you're going to have an issue, that's where it will lie, I'm not sure if others here have had success or (total) failure with that particular card. There should be plenty of space between both heatsinks and DIMMS for those, I doubt you'll have any fitment problems there. Not sure about the case, that one should work but you might be a little squeezed here and there. I assume you read the QVL carefully before selecting that ram. (And even if it's on or off, there's a degree of hit or miss to prepare yourself for there







.)

As I mentioned yesterday I picked up an alternate PSU to see if it fixes my resume problem, and what do you know, it certainly did.



































Finally. Moving forward.

Everything just worked, ANTEC HCP-1000 (pretty nice unit btw, 4x 12V rails). Slept overnight, hit the power button, and it woke right up just like my old ASUS P6X58D-E board. So, it appears that there is indeed some sort of conflict between this board (and a few other ASUS models) and certain power supplies made by CORSAIR or SEASONIC. My old board did not have the problem with the same PSU, so it's something specific to this board and a few others. At this point I'm not sure if the real fault lies with ASUS, CORSAIR, or BOTH. I'll try to contact both and see what they say, I think it will just be finger pointing.









In general though, if you're looking to use S3 suspend and resume with this board, it'd be best to avoid that type of PSU. YMMV.

Quote:


> Only one theory seemed to be plausible, and I have taken quotes from different users explaining:
> 
> "there aprears to be a problem with the +5v rail when resuming from sleep from several Corsair models... I have an AX850 that suffers from it..."
> 
> "The sleep issues had to do with the +5 VSB voltage. I just don't remember the details, but some PSU's (I think a few seasonic models) apparently didn't ramp this up to strict tolerances. But the P67/Z68 boards were the only boards that were this anal about the tolerances."
> 
> "yes, +5 was not ramping up quick enough from sleep...its the chipset (Cougar Point) thats intolerant"


----------



## psyq3212

Ok, here are results of the quick test of Ivy Bridge EP on Z9PE D8 WS:

Please note I am testing QS stepping, which is quite new (they were out short time ago) so it is possible that people using earlier (e.g. ES2) samples could get them to work on BIOS 3506.

- BIOS 3506: No boot, stuck at #00 - indicating broad incompatibility of the BIOS

- BIOS 5004: Managed to boot in Windows (yay!), CPUs properly detected. The only problem is that I cannot get the memory timings to be seen properly in BIOS setup. So I am testing at standard DDR3 1600 frequency until I figure out how to fix this. I am not sure whether this issue has anything to do with the CPUs, as I had exactly the same problem with BIOS 5003 and Sandy Bridge EP

Please note that BIOS 5004 breaks compatibility with old B0 ES samples of Sandy Bridge EP - so people who use them should NOT upgrade. As this is the first BIOS version that breaks those it probably means ASUS performed large update of the code with 5xxx versions of the Z9PE D8 WS BIOS.


----------



## squidman

Wow thanks for exhaustive and very usefull reply! Really appreciate it mate! Been trying to save my gopro vids which got corrupted, so didn't reply earlier (2 days later and 200 euro later trying to reconstruct gopro index files ....don't ask!)I will definately pdf this page and save it. Should be stickied...







Yeah one downside is that the 8-10 cores are coming, but can't see them being cheap (buy e5-2620's now at around 3200 nok or 550 usd or wait? Or just buy one? Iff it even works?

Regardless, thanks again for reallly usefull supply...had a feeling this was the place to go to regarding advice, and I wasn't wrong!







And yes prbly going haf-x case myself, although the lian-li cube thing d8000 doesn't look to bad either..however all that aluminium and not really too friendly just is off putting for some reason...I prefer almost something like the corsair d800/d900, or something where you don't have to use thumb-screws all the time, we should be past that by now....

tight lines! (flyfish, but in computer-speak I guess we could say tight and neat and elegant cable-tying or something!)....


----------



## squidman

This was a reply to blackened tush on my proposed build....thanks!

But on to PS...are there PS and memory sticks that are now pretty much guaranteed to work? Most of the online retailers here are a bit skimpy on details...I'm figuring I need at least 2x 12v rails....and need to buy kit memory that is tested whether 32 GB 1600mhz or 64 gb...i.e. matching...

any other potential pitfalls?


----------



## squidman

Quote:


> Originally Posted by *squidman*
> 
> This was a reply to blackened tush on my proposed build....thanks!
> 
> But on to PS...are there PS and memory sticks that are now pretty much guaranteed to work? Most of the online retailers here are a bit skimpy on details...I'm figuring I need at least 2x 12v rails....and need to buy kit memory that is tested whether 32 GB 1600mhz or 64 gb...i.e. matching...
> 
> any other potential pitfalls?


below is off-topic for blackened tush (mate what a very wrong name! good on ya!)

and yes one more for blackened tush...since you do so much AE and editing...check out prodad mercalli! Wow so much better than warp stablizer....one bad is that you can't save project files, but otherwise...really amazing! Don't know how it does across threads yet tho (still on my puny laptop!) so could be and issue...and the other cool thing is Zbrush and max/maya on a windows surface or eee slate for clay modelling...(most of them are around a grand, running i5)....or better yet you get a samsung q slate thingy...*(samsung ativ Q...the others don't have wacom pens?...and it runs both win8 and android!)....

Ok not into modelling that much yet in zbrush/autodesk alternatives (softimage?) etc, but it does look very sexy! Still want my photorealistic monster stomping through my hood/backplate and causing chaos! With lots of rayfire or pulldownit going on of course (yes max is a buggy mess, but it still has the coolest plugins! Can't go to maya! not without above and realflow and fumefx and krakatoa and and...!!







...

Still on very older versions, hope the 2014 versions have updated (XSI?) something like golaem (maya only as far as I know) or WETA crowd sims (massive)...would still like to see that in max (if not Vue!). Not averse to jumping ship (autodesk for now, started out on c4d), but don't like the gui of houdini or lightwave...not intuitive. Ditto Terragen and a lot of the node-based programs (thanks a lot nuke! for starting that...!). Blechh, the node-thing is ok....and seems pretty intuitive in nuke (good clean gui! but I'm not an expert...) whereas as soon as autodesk attempts a node based thing (hello materials) it's just a friggin' mess...

enough for now, bit tired....srry for long post/rant, was one of those days


----------



## BlackenedTush

lloyd,

After I read that you mentioned switching to your Antec PSU, I started to wonder if _Active PFC_ might have anything to do with it. I looked up the PSU's specs & I didn't see anything about "Active PFC". I knew that the Antec was a very high quality unit. I also had read enough about Active PFC to know that it's something that many of higher-end consumer PSUs (like your Corsair) will promote. But I did see something else that made me go, "HMMMMM..." :
"*Continuous Power*" and "*PhaseWave™ Design - A server-class full-bridge LLC design with a synchronous rectification based on a DC-DC topology"*.

So I thought, "Lloyd's got all that going for him... which is nice... that's a plus...."

But wait...
Thus fueled, a wild & wacky hunch began to drive me.
I did a search for "active PFC hibernate". One set of results brought up a guy that had problem with an old ASUS P5Q motherboard powered by a Power & Cooling S61EPS PSU (_active PFC_). Doing some other similar searches brought up some other sleep issues involving _active PFC_. And so I skimmed a few cryptic tech papers on PSU design and PFC. Now granted, the results of such quick research certainly can't produce any definitive conclusions; but with what I was starting to see, I was thinking, "maybe there is something to this".

On the surface, having Active PFC as a potential negative seems to be highly counter-intuitive, especially considering that it is generally considered superior to passive and a good method of conditioning harmonic distortion out of the output of SMPS PSUs. But I also know that in general, manufacturers have to try to balance quality against costs. Most will wisely cut costs wherever tradeoffs don't produce problems within the wider scope of perceived customer usage. Now as I looked deeper, I found that there are at least a handful of APFC topologies that PSU manufacturers employ when they need that functionality in their designs; such as "SPEIC", "Boost", "Buck", "Buck-Boost", and others. You'll see variations like step-down/step-up, syncronous/asyncrous..,. All have their strengths and weaknesses. It's easy to see that costs would vary between them. Now several, including two of the most common ("Boost" & "Buck-Boost"), produce output that is "dis-continuous". Now it seems to me within my little PeaBrain that such an attribute could be quite significant here. I know that most Corsairs used to be built by manufacturers like Seasonic (which are pretty decent, but they make them for a lot of OEMs). It's obvious that active PFC mileage may vary, There are continuous and dis-continuous varieties. Our Corsair 1200AX could be dis-continuous. But your Antec Platinum claims to be _server quality_. It is very likely that it is indeed a cut above, because it is avoiding the more common, cheaper of the APFC approach entirely, in favor of a design that can use DC-DC rectification. And it outputs "_continuous_" power".

Not being an electronics engineer, it's easy for me to jump all over this _dis-continuous/continuous power_ theory and think there may be something to it. That's the beauty of ignorance. But actually, since our board isn't so robust in areas like the ASM PCI switches and the Marvell SATA chipset, it seems perfectly conceivable to me that ASUS mobo designs are simply not robust in other areas either... and so likely not very robust when it comes to power input... It seems likely that S3 hibernation has never been tested with a wide range of PSU types in general or dis-continuous power specifically.

That said, it almost makes me reframe how I'd categorize this board. Before I bought it, I thought that the references in the documentation to it being a "WS" and "server" board was assurance that it was a predictable, reliable, and stable workhorse... a server board adapted to workstation use.
Then after seeing first hand & hearing second hand how sensitive & finicky it is with certain combinations of components, I still thought of it as a server board; but one that had most likely been rushed to market.
Now I'm wondering if it's referred to a "WS/server" board only because much of it isn't designed as robustly as it could be, and therefore depends heavily upon quality, predictable, _server-grade_ parts to even work!







If true, that would not bug me at all... if the documentation was good enough to let us know these things clearly.

But when you get the Z9PE-D8 working, it really humms...
Well there you have it. Things that make me go _HuMMMM_...

BT


----------



## BlackenedTush

Thanks squidman,

The name _BlackenedTush_ came together based on several on-topic concepts (and some others):

Firstly, I cut my teeth on _Macintosh_ and was a love/hate fanatic of Macs for years. And actually my original concept for this build was a dual-boot _hackintosh_/PC with a single OC'd i7. So I was looking for a deviant name that sounded similar.
Then as I saw the need for cores & started working on this beast, I got several pics in my head... it's such a big mobo & it's got a hulking black case. It reminded me of several automotive projects I'd worked on. I easily pictured looking into the case and then in my minds eye seeing under the hoods of my old MGA 1600 and '65 Ford Mustang - nothing but a sea of black greasy shapes... and I pictured sliding out from underneath them with my back side graced with black oil stains... Lloyd gets the car analogy.
And one of my ancestors immigrated from Norway in 1901. The story is told that as a poor young boy in the swamplands of Minnesota USA, he had only one pair of pants. One day he desperately wanted to go out and play. But he was waiting pants-less while his mother was mending that pair. He kept bugging her to hurry. Finally she said, "Well then, why don't you just _sverte ræva og gå naken_", which roughly means "paint your bottom black and run naked". I always liked that story. And I was similarly impatient to build this beast while it had its early posting, video, and memory issues.
Finally, some perfect metaphors sealed the deal after I'd already named it:
_"it really burned my A**"_ (the frustration I'd felt at my early problems)
_"I really got my A** burned"_ (the fear that maybe I'd spent so much on just a boat anchor)
So yeah.. BlackenedTush.








One day I'll try to add to my list... like maybe eat a lot of beans and get out the matches...








But seriously, I smile whenever I think of the now mighty BlackenedTush and how long it takes to render something at school.

BT


----------



## BlackenedTush

Thanks Jann... good stuff. It's always good to validate things


----------



## psyq3212

Latest info: I also managed to get DDR3 to run @1866 MHz with Z9PE D8 WS BIOS #5004 and two Xeon E5 2697 v2 QS CPUs.

So far so good.

However, the timings had to be auto-read from the SPD, as they are completely messed up in the BIOS setup. Unfortunately, the timings are too low (I guess the SPD read-out was not for 1600 MHz) so after couple of hours I get a machine check exception.

Like I said before, I observed the corrupted timings in the BIOS setup also with BIOS #5003 and Sandy Bridge EP so I am not entirely sure if this has anything to do with the Ivy Bridge EP at all.

Tonight I will also try Z9PE-D16, as I intend to keep that board with Ivy Bridge EP since it has 2x more memory slots. However, Z9PE-D16 does not have memory overclocking control (voltages and such) but it does allow setting DDR3 speed to 1866 MHz. I am going to try flashing the SPD timings and voltage directly on the DDR3 sticks, but right now I am not sure if that works at all and if the board will accept them...


----------



## BlackenedTush

squidman,

About the PSU... The collective knowledge of this forum really is pretty informal and is ever changing. Nobody I've noticed has taken the time to try to compile and verify a blessed list of what has worked for us with what combinations of parts. And what makes it more difficult is that most of the experience here isn't based on scientific tests. And even tho there are lots of very smart people here, we all have difficulty trying to interpret much of the less than stellar manual. And probably a good percentage of us have been burnt trying to get answers from ASUS tech support. I've actually gotten my hard won experience from trying stuff out, from posts in this forum, from the Corsair RAM guy, and a couple of doods named Chris from EVGA support.

When we do think we have good answers, they aren't validated as being correct by ASUS, so your mileage may vary.

You just gotta read a lot and see what has worked for peeps.

I like my Corsair 1200AX active PFC PSU, It's worked very well for me. But then again I disabled hibernation. Now lloyd has the same PSU as I but has his hibernation enabled. He findas that it won't work correctly. But when he recently swapped the Corsair with his server-quality non APFC Antec, that seems to have fixed the problem. But the reasons why this occurs aren't clear yet. I have my hunch; but not being an engineer, I could be off. It would seem to me that if you picked his Antec PSU, you'd probably be making a solid choice all around.

As for RAM, I threw out gobbs of candid opinion early in this topic. My main point that I'd argue 'till I was "black in the tush" is that kits are the only way to go when using today's DDR3 RAM, especially on this complicated board. I'd also say that it doesn't make much sense to get any less than a 4 channel kit, since the spec says that the board supports it.

If you look at the Specifications Summary in the manual (which you should), it looks like you have a choice to populate up to 256GB when using registered RAM; but then you're limited to 1600MHz. And supposedly you are able to OC up to 2133MHz with non-registered RAM if you stay below 64GB (but you'd definitely need good overclocking RAM to do so and I'm not sure I saw any on the QVL (maybe the G Skill). Your mileage may vary. If you do refer to the QVL (like you are supposed to) you may be as confused as I was. First, it is far from exhaustive or definitive. Second, you will see the column heading "Size" which is arguably confusing. Sometimes the RAM listed below it will display a kit capacity like this: "32GB (4x 8GB)". That is great & easy to understand. But sometimes it will say something like "4GB". Which I think I've found to mean the capacity of just 1 stick; but that doesn't tell you the config they tested it with. Was it really just 1 stick? And will an 8 x 4GB kit configuration work? We can only assume. My 64GB kit wasn't even on the QVL. It's Corsair Vengeance quad-channel CMZ64GX3M8A1600C9. The only other Corsair Vengeance RAM I saw were smaller (32GB) kits with slower CAS latency. But mine works well.

Now the Bios has kind of an overclocking pedigree, offering a plethora of settings for CPU and RAM; but it's overkill because it lets you set RAM timing on three levels (but who does that besides overclockers & some gamers?). Plus XEON CPUs are locked. I suppose if you decided to use a single i7 with this board you might wanna OC your CPU then... And with XMP RAM the BIOS detects latency and sets it accordingly; but it doesn't do so with RAM speed. That has to be raised to the XMP rated speed manually. That's a common mobo/XMP quirk I think

There are lots of success stories in this thread you might emulate...
But in summary, here's what I know off the top of my head that _has_ worked:
RAM: Corsair Vengeance 1600 with 9 or 10 CAS latency and 64GB kits or smaller
PSU: ANTEC HCP-1000 Platinum PSU (if you wanna use hibernation) and Corsair 1200AX (if you disable hibernation)

PT


----------



## psyq3212

More news regarding BIOS #5004 and Ivy EP: my GTX TITAN successfully runs at PCIe 3.0 x16 speed when Ivy Bridge EP is used. Card is located at the PCIe slot #3.


----------



## phillydee

BUILD COMPLETE! I'm pretty stoked. Everything is running like it should out of the box--for all the trouble it seemed this board would put me through, I haven't hit a snag yet. Windows 8 installed, and my GTX 680 is working flawlessly. What I've got going:

- Dual Xeon E5-2665 ES QB7U (I believe it's C0 stepping) Speedstep ON, Turbo ON, H/T ON
- 32GB Kingston Unbuffered ECC Server RAM DDR3-1600 KVR16E11K4/32 R(currently running in DUAL CHANNEL MODE, due to 4x8GB sticks. Will probably swap out for a 8x4 set for quad channel)
- BIOS 5004
- MSI GTX 680 Twin Frozor 4GB
- Windows 8 x64 PRO
- Samsung 840pro 512GB SSD
- No raid (yet)

Graphics card installation: because of prior posts stating that this board had problems with GTX6xx cards, I installed Windows using an old Nvidia 9800GT card. I then installed the GForce drivers, then I swapped out the 9800GT for the GTX680. It booted up correctly with the status code of "22", and smoothly went into Windows 8. I'm not sure if this process was a help or not, I just wanted to eliminate all doubt when I was installing Windows.

Another thing to note was that with the C0 stepping chips, someone mentioned that VT-d is not supported. I made sure to DISABLE this after the 5004 bios update, as it's set to ENABLED by default.

Cheers


----------



## anubis1127

Great, that's awesome that you got it all together successfully.

Any pics?


----------



## BlackenedTush

Awesome phillydee!


----------



## BlackenedTush

psyq3212,
That's great information... and I like how you fill in the pertinent details


----------



## BlackenedTush

phillydee,

I may just have learned something... I hadn't really thought it thru before; but it makes sense that to run in quad channel you'd either need to have one CPU with 4 sticks or two CPUs with 8.

BT


----------



## phillydee

Quote:


> Originally Posted by *BlackenedTush*
> 
> phillydee,
> 
> I may just have learned something... I hadn't really thought it thru before; but it makes sense that to run in quad channel you'd either need to have one CPU with 4 sticks or two CPUs with 8.
> 
> BT


You're right. For me, this was the recommended configuration according to the Asus manual. I didn't really feel like getting experimental right now; I'm just happy I'm half way through re-installing all my necessary applications.

Here's a thought... buy two sets of the same 4x8GB kit and use the quad pair on each CPU... so all RAM assigned to it's CPU socket are from a matching pair. Could work with ECC/server grade RAM.

Pics are coming! (eventually)


----------



## brock2621

Quote:


> Originally Posted by *phillydee*
> 
> You're right. For me, this was the recommended configuration according to the Asus manual. I didn't really feel like getting experimental right now; I'm just happy I'm half way through re-installing all my necessary applications.
> 
> Here's a thought... buy two sets of the same 4x8GB kit and use the quad pair on each CPU... so all RAM assigned to it's CPU socket are from a matching pair. Could work with ECC/server grade RAM.
> 
> Pics are coming! (eventually)


Really wish you woulda filmed this, I'm about to go through setup in a few days and I have no idea what to change in the bios from its default.


----------



## BlackenedTush

squidman,

Like Jann said, the HAF-X is tight - width wise anyway...

The Z9PE-D8 board just barely fits in and the right CPU covers the wire management pass-thru that would be usable with an ATX board. There are slots you can use between the drive cages and the board, so it's really not a problem except for when replacing RAM on the right side (if you have the optical bays filled like I do). I needed to replace my RAM after install. To do so I had to disconnect a few SATA cables from their backplanes before I could get free access, but it wasn't too bad. Here are some pics...

View of tightness between board and drive cages (with RAM installed & cables connected):


View of slots that are usable for routing cables (where orange light shines thru)


BT


----------



## phillydee

Quote:


> Originally Posted by *brock2621*
> 
> Really wish you woulda filmed this, I'm about to go through setup in a few days and I have no idea what to change in the bios from its default.


I'll try and snap a pic for you next time I boot up. The only meaningful change I made was the VT-d setting from Enabled to Disabled. That's only because I have an ES type chip--if yours is retail, you don't need to do what I did.

Other things I did were the usual--no boot logo, no 3rd party RAID hardware enabled etc. That kind of thing.


----------



## lloyd mcclendon

RE: power supply quirks, I'm not really sure either, I've passed the issue along to both ASUS and CORSAIR for their benefit, who knows if the right words will land in front of the right eyes, probably not.

The bios defaults are actually pretty good, the only things I touch (AFAIK) are: disable boot logo, disable marvell controller, enable VT-D, x2APIC, link training timeout, enable power on by PCIE, disable APSM, both QPI links enabled (not sure about effects ...), and force 1600MHz.. It'd be nice if I wrote these down or better yet if we had the OC profile feature other ASUS models have.

Nice to see a run of positive results in this thread, back on page 2 or so it was all:














v5004 looks like they're getting it together.

I don't really follow intel's upcoming stuff but I see now we have this "2697" next which is apparently is a 12 core but a bit slower than the 8 core 2687w. Interesting. I was so excited to hear of a new chip for the 1% chance that it was unlocked. Nope, but nice to see that 2011 isn't dead already.







Is anyone aware of an unlocked 2011 xeon in the works or at least _rumors_ of any? if only...


----------



## BlackenedTush

Quote:


> Originally Posted by *phillydee*
> 
> Here's a thought... buy two sets of the same 4x8GB kit and use the quad pair on each CPU... so all RAM assigned to it's CPU socket are from a matching pair. Could work with ECC/server grade RAM.


I've known that this board uses a NUMA server architecture... one aspect of which basically supports each Xeon's integrated mem controller's ability to control it's own ("near") memory and to also share it across the QPI backbone to the other CPU as ("far") memory. The mobo has a mem controller for each RAM slot to help facilitate this.

I've sort of characterized that complexity as a weak point because it's not clear how much of Intel's 602 chipset ASUS uses for that functionality and how much they've built on top of it. I do know that they've tacked on USB 3.0, 1394a, some creative PCIe switching, Marvell SATA & RAID, Realtec Audio, etc. to make this board more marketable to consumers. And some of that stuff inn this implementation has had its problems. So as a result I've whined a lot about the board's finicky sensitive side. But to be fair I've never seen it proven that how this board treats RAM is any more finicky than most other boards. And I haven't proven the mixed kit thing is really a problem either. I _do know_ that it works crappy with _bad_ RAM tho. Because the Corsair RAM tech peeps strongly urged me to not mix kits on this complex of a board, I've carried that torch. But maybe that's not completely fair. It could be that the NUMA architecture with Xeons is actually more tolerant than I think. Maybe the worst that could happen mixing kits of the same type is some performance degradation. So all of that to say this...

You are probably right. Two different ECC RAM kits of same type (especially registered) would probably always work perfectly fine. That's what they're built for. To be fair again, ever since I got my board running smoothly, it has shaped up to be a great workstation.

BT


----------



## brock2621

Quote:


> Originally Posted by *lloyd mcclendon*
> 
> RE: power supply quirks, I'm not really sure either, I've passed the issue along to both ASUS and CORSAIR for their benefit, who knows if the right words will land in front of the right eyes, probably not.
> 
> The bios defaults are actually pretty good, the only things I touch (AFAIK) are: disable boot logo, disable marvell controller, enable VT-D, x2APIC, link training timeout, enable power on by PCIE, disable APSM, both QPI links enabled (not sure about effects ...), and force 1600MHz.. It'd be nice if I wrote these down or better yet if we had the OC profile feature other ASUS models have.


Sorry for the Noob questions but its been 6 years since I last built a PC and dual core AMD's were top of the line back then just to give reference to how long it's been haha:

First, Power Supplies:

I've got two power supplies I keep going back and forth between after reading through this forum:

A: Silencer Mk III 1200

B: Ax1200i

Which would you recommend?

Second, Bios Settings:

1. Disable Marvel Controller - I assume you aren't running any types of RAIDS? What the benefit of disabling?

2. Enable VT-D - This is on by default in newest bios right?

3. x2APIC - What is this? Enable, disable?

4. Link Training Timeout - What's this?

5. Enable power on by pcie - Would this be for waking up via network card? What would this benefit?

6. Disable APSM - What's this and why?

7. QPI Links Enabled - What's this?

8. Force 1600 - I plan on 64gb, 8x8. Why would the board default slower?

Your answers will be fantastic (or a cell phone pic would be awesome too), so that way I can boot up and easily run through the settings and be up in running really quick.

Also, I'm getting a Titan and I've heard slot 1 and then slot 3, which one should it go into?

It would be very nice to have a sticky that had optimum bios settings and an unofficial QVL for stuff too









Thanks so very much!


----------



## lloyd mcclendon

comments inline ... you're welcome. Anyone else please chime in with bios setting changes we should be considering
Quote:


> Originally Posted by *brock2621*
> 
> A: Silencer Mk III 1200
> B: Ax1200i
> Which would you recommend? *If you don't want to use suspend, you could try your luck with either one. If you do, the second one is probably a no go ; but otherwise its a terrific unit. See my previous 3 or 4 posts about it*
> 
> 1. Disable Marvel Controller - I assume you aren't running any types of RAIDS? What the benefit of disabling? *Software raid - why anyone would run hardware raid and tie the life of their data to the life of the motherboard is beyond me.. The marvell chip is as usual incompatible with VT-D*
> 
> 2. Enable VT-D - This is on by default in newest bios right? *Yes .. if you're not passing pci cards to a virtual machine you don't need it.*
> 
> 3. x2APIC - What is this? Enable, disable? *Not really sure, but the linux kernel has an option for it, and it's on by default and seems like good stuff.*
> 
> 4. Link Training Timeout - What's this? *Seems to help fix (or workaround) an issue with the board where the card may not always be detected on boot.. YMMV*
> 
> 5. Enable power on by pcie - Would this be for waking up via network card? What would this benefit? *Yes, wake on lan*
> 
> 6. Disable APSM - What's this and why? *Needed for S3 suspend compatibility apparently, check the knowledge base on asus's site under the z9pe support page*
> 
> 7. QPI Links Enabled - What's this? *Not really sure ... would love to know. nobody here does
> 
> 
> 
> 
> 
> 
> 
> I'm experimenting with it, stability is fine, not sure if any tangible difference.*
> 
> 8. Force 1600 - I plan on 64gb, 8x8. Why would the board default slower? *Don't know, that's what it it does, it will show up probably at 1333*


----------



## psyq3212

QPI Links should be enabled as Sandy Bridge EP / Ivy Bridge EP CPUs are using the QPI links to communicate with other CPUs installed on the system and send/receive results from the remote NUMA memory.

In the Z9PE D8 WS there are also settings for enabling/disabling L0 and L1 states - these are power saving states and they should be kept on default values (some ES CPUs also had issues with entering power save states and for them both states should be disabled, but do this only if you have stability issues).

If all QPI settings are left @default values of BIOS #5004, it is OK.

As for "Force 1600" - normally, unless you are running "overclocker" RAM with XMP profiles if you leave the DDR3 speed at "Auto", Z9PE D8 will detect the SPD timings and operating frequency properly and there is no need to set a particular frequency manually. However, if you are sure the board is not detecting your DRAM configuration you can override the frequency and also timings and voltages. Normally, there is no need to do this with server-grade ECC RAM, unless you want to overclock it.

I have no issues running my Samsung ECC RAM (1600 MHz, 1.35v) at 1866 MHz with just bumping the voltage to 1.5v but YMMV. If you want absolute rock-solid stability and do not want to experiment and test tighter timings or higher RAM clock, just leave everything in memory configuration at default values. Do not disable NUMA mode in any case.

As for the TITAN card - for C0/C1/C2 stepping Sandy Bridge EP, it should not matter whether the card is in the Slot #1 or Slot #3 - however, certain people (myself included) had issues when the card was in Slot #1 (no POST). So, just try putting the card in Slot #1 and if the board does not POST, move it to Slot #3. Please note that for enabling PCIe 3.0 x16 speeds you would need to patch the registry (either manually or by using NVIDIA command line tool for forcing PCIe Gen 3). Leave the link training timeout to 5000, which is a new value introduced in BIOS version 500x.


----------



## BlackenedTush

Quote:


> Originally Posted by *lloyd mcclendon*
> 
> The bios defaults are actually pretty good, the only things I touch (AFAIK) are: disable boot logo, disable marvell controller, enable VT-D, x2APIC, link training timeout, enable power on by PCIE, disable APSM, both QPI links enabled (not sure about effects ...), and force 1600MHz.. It'd be nice if I wrote these down or better yet if we had the OC profile feature other ASUS models have.


Or maybe take pics. I did that once; but then changed the settings and never took updated ones...









I don't know what you mean by "both QPI links enabled". I see _QPI Link1_, _QPI Link0s_, and _QPI Link0p_. Maybe my bios version is way outdated.

Quote:


> Originally Posted by *lloyd mcclendon*
> 
> ...nice to see that 2011 isn't dead already.
> 
> 
> 
> 
> 
> 
> 
> Is anyone aware of an unlocked 2011 xeon in the works or at least _rumors_ of any? if only...


Around the time of the release of these CPUs, people who seemed to be in the know talked a lot about Intel taking a different approach to making chip improvements. It was said that they're making Xeons less desktop/consumer oriented and more server oriented - to target the growing demand for data centers. That meant less focus on what they used to do (increase clock speed and circuit size), and put more focus on efficient data movement. So they integrated I/O, PCI Express 3.0, networking, and improved core and memory interconnection with QPI, etc.. And of course they locked it down. These Xeons are full blown server pedigreed CPUs. I could be wrong but I wouldn't expect a K version of Sandy Bridge Xeons, because it looks like Ivy Bridge models are going further down that same road. They are mellowing out the clock speeds and upping the top number of physical cores offered to 12. In fact if you hold an empty CPU box up to your ear, I think you can hear the faint echo, "we upped ours, so up yours"...








We can always hope some traditionalist Intel engineers will go rogue and start up the Intel Deviant OC Racing division or something.


----------



## psyq3212

Decision not to unlock Xeons is simply a business one.

Overclocking has become more popular in server market (high frequency trading, etc.) and unlocked 2S EP Xeon would potentially cannibalize sales of 4S systems. Unlocked 4S Xeon might even cannibalize some EX sales as well.

Of course, this is just my opinion, but I would assume that the unlocked Nehalem / Westmere Xeons were just an anomaly - with Nehalem platform Intel wanted to settle the score once and for all regarding being able to innovate, which they started with Core 2 platform - this is probably why they launched Nehalem with HEDT platform.

Therefrore I doubt we will see unlocked Xeons in the future.

There were some rumors that Intel is assembling the old Skulltrail team to do something with Haswell EP similar to what they did with Skulltrail - take the 2S Xeon platform and make an uber-high-end dual-CPU workstation/enthusiast setup. But I do somehow doubt that this will happen (although I would definitely like to see that), I see no reason why Intel would need this kind of marketing exposure today, while in the Skulltrail times, they really needed to show that they are far ahead of AMD.

So, until we see if this is really going to materialize with Haswell EP, with Ivy Bridge EP we are stuck either with HEDT crumbles (castrated EP Xeons with 6 cores enabled) or with completely locked EP Xeons which can only be overclocked by bumping BCLK few MHz up.


----------



## squidman

Blackened Tush,

Thanks for the pics...yeah looks a bit tight...may actually spend the buckaroos (or kroner in this case) and opt for the monster corsair d900. Great norwegian-american analogy...remember one of our Minnesota builders who spoke unintelligable english, in the worst Fargo way, but then didn't speak a word of norwegian!

Will probably go with the corsair 1200ax (or was it crucial? still struggling with corrupted sdxc card here...). May hold off on 2x e5-2620's, just plop in one until the xeon v2's come out (8 and 10 core), although no doubt they're out of my price range (less than 5000 NOK or 1000 USD each - 25% VAT and markup sucks!).

I assume cpu's need to be matched (sorry newb question...but what if they run at the same frequency?)...









Go the lutefisk!


----------



## phillydee

Hey still in the middle of tidying up and setting up new office and furniture, equipment etc. but had this naggy little question... it's baffled me for a bit here so:

- when in Windows 8 and I tell the system to sleep, it does it's normal routine, screen blacks out, the fans go quiet... and then just when it seems like power is suspended, the hardware starts up again. It's a headscratcher. Maybe I have my case ATX power button plugged in wrong? I can't seem to figure this one out.

System's so fast that I just opt to shut down right now--but any suggestions are always helpful. Thanks!

BT: read through your ram writeup a few pages back and it's given me some thoughts. No time right now but if I do decide to go two sets of the same ECC RAM, I'll be sure to come back here and log my experiences.


----------



## BlackenedTush

squidman,
Quote:


> Originally Posted by *squidman*
> 
> ...may actually spend the buckaroos (or kroner in this case) and opt for the monster corsair d900.


Yeah... I was drooling over that case. Someone mentioned thumb screws being antiquated... anyway, it kinda looks like Corsair heard them and was planning to do some advanced kind of quick-connect design on their PCI slot covers, but then stopped after getting the covers fabricated. Then they decided to keep using the old screw method with the new fabrications.

Hey, I found some information on mixing CPUs in the Intel datasheet: _Intel® Xeon® Processor E5-1600/E5-2600/E5-4600 Product Families Datasheet - Volume One_, in section 7.5, called _Mixing Processors_.
It says that all processors must operate with the same:

QuickPath Interconnect (QPI) frequency
core frequency
power segment
internal cache sizes
The Intel site has a lot of good info on processor models and provides the ability to select and compare them.

I definitely wouldn't want to be required to buy my annual Lutefisk Cod from some Fargo Minnesotan dude who I couldn't understand... while he's tacking on a 25% VAT. That would be the ultimate bad day. haha
Seriously tho, I like Lutefisk. It brings back good memories of beloved family members from the old country soaking the stuff in lye and wrapped in stinky old undershirts.

My power just went off and my UPS kicked in. I was computing on battery power for 10 minutes. Although I could no longer connect to any _new_ internet sites, I barely noticed. Yeah Cyber Power!









BT


----------



## psyq3212

Basically what Intel is saying - they only test and validate under conditions stated above.

In theory, it might be possible to get other combinations of mixed CPUs working, but this is a matter of luck and can be dependent on BIOS, microcode / MRC version etc. etc. etc.

So, unless someone is getting the CPUs for free, I'd not do something which is out of what is described in EDS spec.

By the way - I got the Xeon 2697 v2 working in Z9PE-D16 as well. Same as Z9PE-D8 WS, the latest BIOS is required.

Other than that, I have a funny issue with IVB-EP - HWInfo64/CPU-Z report that the memory is operating in dual-channel mode (despite the fact that all the DIMMs are populated).

I am not yet sure is this really correct or HWINFO/CPU-z are misreporting memory configuration due to some incompatibilities with Ivy Bridge EP, or is it an incompatibility of ASUS BIOS MRC with the C0 stepping of IVB-EP.


----------



## BlackenedTush

phillydee,
Quote:


> Originally Posted by *phillydee*
> 
> - when in Windows 8 and I tell the system to sleep, it does it's normal routine, screen blacks out, the fans go quiet... and then just when it seems like power is suspended, the hardware starts up again. It's a headscratcher. Maybe I have my case ATX power button plugged in wrong? I can't seem to figure this one out.


It's a little baffling but that seems like a logical thought... not sure it would be easy to rule that out or not because it could be a little more complex... being that it's connected to the Front-Panel header. And it amazes me that more people don't have problems with that aspect of a build... all these little connectors on adjacent pins. There are several potential failure modes there.
But it wouldn't hurt to disconnect the FP Power Button and try.
Waddaya think... Maybe open your case, disconnect just that pair of wires from the front-panel header, start it up again from the built-in switch on the board, and then try putting it to sleep again with Win8).
If you are on the right track here, it could be that there are more wires off than just those two. So it wouldn't hurt to just go back and check the position of all of those FP wires as best you can. If I remember correctly, my board came with a little intermediate connector for that header. It orients the front panel wires correctly and then plugs into the header. That made getting connectors on the right pins a little easier. Did yours have that?
Quote:


> Originally Posted by *phillydee*
> 
> BT: read through your ram writeup a few pages back and it's given me some thoughts. No time right now but if I do decide to go two sets of the same ECC RAM, I'll be sure to come back here and log my experiences.


That would be great.

BT


----------



## phillydee

Quote:


> Originally Posted by *BlackenedTush*
> 
> phillydee,
> It's a little baffling but that seems like a logical thought... not sure it would be easy to rule that out or not because it could be a little more complex... being that it's connected to the Front-Panel header. And it amazes me that more people don't have problems with that aspect of a build... all these little connectors on adjacent pins. There are several potential failure modes there.
> But it wouldn't hurt to disconnect the FP Power Button and try.
> Waddaya think... Maybe open your case, disconnect just that pair of wires from the front-panel header, start it up again from the built-in switch on the board, and then try putting it to sleep again with Win8).
> If you are on the right track here, it could be that there are more wires off than just those two. So it wouldn't hurt to just go back and check the position of all of those FP wires as best you can. If I remember correctly, my board came with a little intermediate connector for that header. It orients the front panel wires correctly and then plugs into the header. That made getting connectors on the right pins a little easier. Did yours have that?
> That would be great.
> 
> BT


No intermediate connector... I know what you're talking about; those are really handy! I mean, maybe I didn't inspect Asus' packaging correctly. I'm pretty uptight when it comes to only opening the packages that need to be opened, even though I feel like a kid at christmas heh.

Back to the power button issue: so my case front panel ATX switch only has 2 pins. On the motherboard, the contact for the ATX button has 3 pins, the POWER being in the middle of two neutral pins. So, right now I've got it plugged in to the LEFT of the power pin. I could try swapping it to the right pin. When I first plugged in my PSU and then threw the PSU switch to on, the whole system started powering up which could be an indication that the board thinks contact's being made. Anyway.

BTW Windows 8 pro's got some hefty caveats. Wrong discussion thread for that but yeah. UAC. Such a PITA.

(edited: I had previously thought the middle pin was neutral. The outer two were, so I edited that to reflect change)


----------



## phillydee

Some pictures from the build: Just got the first H80i mounted, getting ready to do the next one. This case for how large it is really doesn't facilitate two H80i coolers very well. I had a tough time figuring out where to put them.


Shortly after I got the system to POST. I'm about to assemble everything else.


Samsung 840Pro 512GB for my system. The COSMOS II's redeeming feature for me was it's mounting holes for SSDs.


This gap!







is caused by the H80i's radiator being mounted on top, above the fan vent hole as seen below


I'm sure a more "professional" builder would have done things differently. I almost returned the case because of this problem. I couldn't afford the downtime unfortunately.


Everything in, along with my GTX680.


Cheers


----------



## phillydee

*SOLVED: SLEEP MODE*

Two issues slain. I'm pretty happy, as I'm about 99% there and ready to get on with this system. First, I reversed the power plug pin, that solved the issue of the system turning on immediately when I plug in to the wall.

For the Windows 8 Sleep issue: I had to turn off "Wake On LAN", and that solved it. I also have Hybrid Sleep disabled (in advanced power settings).


----------



## lloyd mcclendon

So, does it resume with your AX1200? If so when did you purchase it? Mine is about 3 years old. We should compare ...


----------



## psyq3212

One note to people who plan to purchase new IVB-EP Xeons: the HCC (High Core Count) versions (with 12 cores) come in a slightly larger CPU package.

Picture below shows SNB-EP (2687W) on the top and IVB-EP (2697 v2) HCC on the bottom.

As you can see, 2697 v2 is noticeably wider! However, it will still fit in the LGA2011-0 socket *but* you should be very careful how you insert it, as it is a bit more hart to align it properly without the insertion tool. So, be careful when you are putting the HCC versions in to avoid bent pins.


----------



## TheBlademaster01

Nice, thanks for all of the info psyq. Did you buy them from eBay? If you have any leads on other IVB-E chips please let me know









What is the Q-Spec of that 2697 QS?


----------



## psyq3212

No, it was not from eBay but if I have any leads I will certainly tell









As for the sSpecs (actually, it is QDF since they are not-retail) - mine are QEEY.

QEEY corresponds to C0 stepping of Xeon E5 2697 v2
QF93 corresponds to C1 stepping of Xeon E5 2697 v2

Both C0 and C1 are Qualification Samples (QS) so they represent the future shipping SKU in features. The differences between C0 and C1 are small - unfortunately I cannot write what the difference is publicly, but there is (at least) one item that was not fixable by microcode update / MRC workaround and had to be fixed with new C1 stepping, however I would doubt this particular issue would be encountered in normal workloads, so I would say C0 are safe for "home" use - if you could get hold of them.

I suppose Taiwanese/Chinese eBay sellers are waiting for the platform launch in order to avoid attention which they would certainly get if they start selling IVB-EP before official launch. After the launch I suspect that they will offload B1/B2 (ES2) samples first because I suppose this is what they usually get in this stage in the gray market









I do not recommend buying B1 stepping in any case, as it has several features disabled due to issues. B2 stepping is somewhat better - QDF for 12-core part is QDKA for those interested. However, as usual with ES2 samples, its clock is lower few bins than the final Xeon 2697 v2 spec, and it is not guaranteed that all boards will support them - and, they might drop support for them in the future like it happened with B0 Sandy Bridge EP-s.

Also, please note that 8/10 core SKUs and 4/6 core SKUs will have different steppings (M0/1, L0/1, R0/1, S0/1, etc..)


----------



## TheBlademaster01

Thanks again man, good to have you around in this thread









I have B0 chips now but when I get IVB-E chips I'll just buy a new Bios chip and replace it with latest BIOS. I will make sure to get a C stepping this time though


----------



## BlackenedTush

psyq3212,
Quote:


> Originally Posted by *psyq3212*
> 
> Basically what Intel is saying - they only test and validate under conditions stated above. In theory, it might be possible to get other combinations of mixed CPUs working, but this is a matter of luck and can be dependent on BIOS, microcode / MRC version etc. etc. etc.


I totally agree. But the majority of us arguably range from first-time builders who's bravado exceeds their knowledge, to highly experienced builders who aren't quite computer scientists in the hardware arena. So like you say, "unless someone is getting the CPUs for free", and I would add, unless someone has lot's of time to just play, it's best to stick to combinations that Intel will support.
Quote:


> Originally Posted by *psyq3212*
> 
> By the way - I got the Xeon 2697 v2 working in Z9PE-D16 as well. Same as Z9PE-D8 WS, the latest BIOS is required.


Sounds like you are getting these things for free and have time to play... haha
Quote:


> Originally Posted by *psyq3212*
> 
> Other than that, I have a funny issue with IVB-EP - HWInfo64/CPU-Z report that the memory is operating in dual-channel mode (despite the fact that all the DIMMs are populated).
> I am not yet sure is this really correct or HWINFO/CPU-z are misreporting memory configuration due to some incompatibilities with Ivy Bridge EP, or is it an incompatibility of ASUS BIOS MRC with the C0 stepping of IVB-EP.


Yeah, it's hard to say what's really going on; but the most reasonable hunch IMHO is that CPUZ is simply on the hind end of the development curve and that a later version will fix things.

I do appreciate people like you who have what it takes to experiment and who also take trouble to document their adventures... It's of collective benefit. Thanks

BT


----------



## psyq3212

Yes, QS samples are generally much better idea than ES2 (not to mention ES1, which is even more risky).

However, if it turns out that only B1/B2 samples are widely available in the near future on eBay (4P Ivy Bridge EP Xeons also have B3 ES2 stepping!), I'd suggest to wait until somebody tests them on popular boards like ASUS Z9PE-D8 WS, as anything might happen with these once Intel stops supporting the ES2 samples in the MRC.

Good news is that current Z9PE D8 WS and Z9PE D16 BIOS obviously have beta microcode and MRC for Ivy Bridge EP, which also means that, in theory, ES2 samples should also be supported (support for ES1/ES2 samples gets dropped in production microcode update and MRC from Intel). But "in theory" does not mean that there are no other quirks, like what happened with B1 Sandy Bridge EP which generally refused to work in 2P configuration in ASUS and Supermicro C60x boards.

--

By the way I also tested memory bandwidth now - there is no way that it runs in dual-channel mode, so I suppose HWiNFO64 and CPU-z are just misreporting the memory configuration as "dual channel" instead of quad channel.


----------



## BlackenedTush

phillydee,
Thanks 4 the pics. Compared to my HAF X, the Cos is luxuriously roomy around the mobo. I love seeing progress.
At the risk of stating the obvious (but considering that when in the throes of building, there's a lot a stuff going on in one's mind and maybe less patience), things easily get missed that wouldn't otherwise... did you try putting the rad in the case and the fan outside? It just looks to me like the fan is thinner.... something a quick measure would verify .
Flipping it and checking it are the kinds of heroic activities that push peeps up into the stellar ranks of the "professional builder" you were talking about. haha
BT


----------



## BlackenedTush

psyq3212,
Have you worked for Intel?
Bt


----------



## phillydee

Quote:


> Originally Posted by *BlackenedTush*
> 
> phillydee,
> Thanks 4 the pics. Compared to my HAF X, the Cos is luxuriously roomy around the mobo. I love seeing progress.
> At the risk of stating the obvious (but considering that when in the throes of building, there's a lot a stuff going on in one's mind and maybe less patience), things easily get missed that wouldn't otherwise... did you try putting the rad in the case and the fan outside? It just looks to me like the fan is thinner.... something a quick measure would verify .
> Flipping it and checking it are the kinds of heroic activities that push peeps up into the stellar ranks of the "professional builder" you were talking about. haha
> BT


I tried to put the rad on the inside. However, I was extremely worried about not being able to plug in the ATX power plug. There really didn't seem like there was any room at the time... I think eventually I'll re-seat the contact and when I do I'll try it again (with everything already plugged in). Either way, I learned a lot working with two H80s as I had never used any closed loop coolers prior to this build. One would think there'd be enough room... I could've been just too cautious.


----------



## psyq3212

Quote:


> Originally Posted by *BlackenedTush*
> 
> psyq3212,
> Have you worked for Intel?
> Bt


No


----------



## psyq3212

I am getting occasional BSOD which I originally thought had something to do with the CPUs / microcode.

However, after analysis of the Windows event logs, it looks the BSOD was coming from repeated errors from the NVIDIA driver (Event ID #14, which Windows complains it has no description, and it comes from nvlddmkm.sys). This error starts piling up several times at once, and in frequency of several hours. After the number of error reaches some higher count, Windows bugchecks with a MCA exception.

As I patched the registry to force PCIe 3.0 x16 speed of the TITAN card, I now reverted to the PCIe 2.0 speeds to see if the issues are going to persist - as it might have something to do with PCIe training.

I also noticed that the ASUS BIOS #5004 does not contain the latest microcode update for the IvyTown CPUs (built-in version 40C), and I know that the several subsequent beta microcode updates solved quite a lot reported sightings, so usage of Ivy Bridge EP with Z9PE D8 WS is still a "beta" experience.


----------



## psyq3212

OK, several hours of torture and not a single error in Event Log.

So, forcing the link to PCIe 2.0 did solve the issue. Now I have to figure out is it CPU-related or NVIDIA driver related, as I just found out that people did complain, in fact, about the same Event Log error and BSOD on NVIDIA forums, and somehow this is related to their recent drivers...

Funny times...


----------



## brock2621

Hmm... Interesting psy. I will be following your posts as this is about to directly effect me. I'm picking up this board and a Titan on Monday


----------



## AndyE

Quote:


> Originally Posted by *BlackenedTush*
> 
> Here's a thought... If the board isn't too touchy with 3GB GTX 660s or a Titan (I don't know the latest on compatibility and the BIOS situation), two 660s would supply the same number of CUDA cores & RAM to Octane, AE & Premiere as a Titan GPU costing double that. That's 2688 CUDA cores and 6GB of RAM. There may be single cards or dual setups that will hit your value/performance sweet spot better.


The board is stable with Titan's under prolonged load (weeks). Did run up to 4 of them in one system. Even with PCI Express 3.0 x16 speed settings.
Quote:


> *Dual Xeons vs OC'd i7*...
> The Xeons with their superior number of cores will be all around workhorses that will outperform the single CPU in the widest range of 3D and rendering situations. It is possible that a fast, single multi-core CPU like an OC'd i7 could edge out a pair of slower E5-2620 Xeons in rendering if you're using Octane and a couple of fast CUDA GPUs. It's hard to say tho without testing. And I'm sure that in general Windows performance, the OC'd single CPU would feel snappier with the fast i7 than with a pair of Xeons


The biggest difference between a dual Xeon and OC/i7 system is the available memory bandwidth. If your application is CPU bound, then the OC way might be more cost effective vs dual 2620, if your applications are rather memory (bandwidth) bound, go dual Xeon - even with lower core count.

My initial post in this thread was #180 for more info, if interested. A pic of the config.


----------



## AndyE

psy,
thank you very much for all the useful IB EP information. Appreciated
I am waiting for E5-2687 v2 version with 10 cores, which in my back-of-the-envelope calc will be the highest performing one in the lineup


----------



## TheBlademaster01

Question

I'm running dual 2665s ATM @2.5GHz (slight overclock) and getting 18.61 in CineBench R11.5. Is this correct?

This is what I see in most reviews



Also, how can I see whether my RAM is in Dual vs Single channel in Ubuntu?


----------



## Torwar

Best bios for 2687w es C0 stepping ?
Have now tried it with the card had when i buy it last week 3506 and latest 5004
3506 give suddenly shutdowns and 5004 is difficult to start work only nr3 pci-e give picutere and now it only runs with 1 CPU.
Is one of the old bios good ?
I have no use for sli ore crossfire one card is ok for me.
Memory i use is Kingston DDR3 1600MHz 8GB ECC
ValueRAM CL11 DIMM w/ TS x4=32 total

Use wil be computerchess Deep fritz 13 run in 2d so i most cpu use but i wil use comp some computer gameing to.


----------



## TheBlademaster01

When using a single CPU only 4 of the PCI-e slots are on IIRC


----------



## Torwar

I have 2 cpu but card only wil boot up with 1 cpu on the 5004 bios.
i have read many post so understud since i have ES cpu i have to use a old bios but who run ok ?


----------



## TheBlademaster01

I believe C0 will work with the latest BIOS version. B0s don't


----------



## Torwar

The card wil not boot any more so i return it to dealer were i buy it.
Look like it some serious wrong with it onboard bios led light is dead and when i try to start it so run fans 2 sek and stop.
Is it any other mobo who runs with ES co cpues ?


----------



## TheBlademaster01

Supermicro X9DAi, X9DRL

Asus Z9PE D16

I think almost all boards run C0 ES


----------



## Torwar

Thanks for Answer TheBlademaster01


----------



## TheBlademaster01

No problem









You might want to wait for some response of the more knowledgeable members though. I'm also in the process of resolving an issue


----------



## psyq3212

BIOS #5004 should work with SNB-EP C0.

The problem Torwar has is PCIe 3.0 x16 training related - my theory is that pre-C2 SNB-EP CPUs have problems with NVIDIA Kepler GPUs during PCIe link training. I think that full PCIe 3.0 compliance was not achieved before C2 stepping and this is why HEDT Sandy Bridge 3930/3960x did not have PCIe 3.0 as advertised. Production Xeon E5 were mostly C2, so the problem had been probably fixed before the large batches of Xeon E5 CPUs were produced. But people who have either C1 Core i7 or C0/C1 QS Xeon E5 might have issues with some PCIe 3.0 devices.

I am afraid this problem has not been really solved yet. ASUS does not make it better by not allowing to force the training to PCIe 2.0 speeds, so in the worst case you are stuck with the loop of unsuccessful PCIe 3.0 training attempts until one of them work.


----------



## TheBlademaster01

Hi psyq

Can you see what's wrong here? I have 4 sticks of RAM and I tried them in various combination but I'm fairly positive that my RAM is not in dual channel.

2 sticks in - 2x 4GB

http://browser.primatelabs.com/geekbench2/2188069

4 sticks in proper order - 4x 4GB

http://browser.primatelabs.com/geekbench2/2186672

CineBench makes no difference. Both 18.6 points.

I'm running 2x E5-2665 @2.5GHz overclocked


----------



## psyq3212

Hi TheBlademaster01,

Can you run Windows? If you do not have Windows installation, you can just copy the contents on the Windows installation ISO to a USB stick and boot into the Windows setup, then go to "recover my computer" option.

Also, copy the contents of HWiNFO installation ZIP file (http://www.hwinfo.com/)

Then, after you boot Windows (or Windows setup -> "Repair my computer" -> Command Prompt", launch the HWiNFO.

Then go to memory summary - it will display the memory configuration (single / dual / quad channel)

Please let me know if it helped.


----------



## brock2621

Aight, I got everything in today except the dual 2687W's. Running through a few things and making sure I understand everything before I jump into it tomorrow. Some of these are serious N00b questions so bare with me and I'd love any/all the help I can get.

1. Which of the SATA ports should I plug my 840 pro 512 into? One of the 4 gray sockets at the bottom right?

2. ASUS has a crap-ton of updates on this page (http://www.asus.com/Motherboards/Z9PED8_WS/#support_Download_30); should I download all of them and install?

3. Can anyone post their bios screens for recommended settings (what things to turn on and off?) For instance, I'm only running a single 512 SSD so I wouldn't need marvell raid crap on right?

4. Any board jumpers I should go ahead and turn on/off that are different from their shipped default?

5. I think I'm going to end up covering up the GPU slot 1 with my CPU coolers so I will probably end up putting the TITAN on #3. Is there a difference?

I'm sure I'll think of lots more but for now I appreciate any help I can get. Thanks so much!


----------



## psyq3212

1. Unless you are using SAS devices, you should use Intel 6 Gbit/s SATA ports (SATA #1-2 blue are 6 Gbit/s). After those two are populated, ti depends on the drivers you'll be using. With Marvell's drivers I am getting decent performance for other SSDs from Marvell 6 Gbit/s SATA ports (gray). However, if you use Windows built-in Marvell driver the results might be worse than using Intel's SATA 3 Gbit/s.

2. If you are using Windows, I recommend to update to the latest Intel RSTe (SATA) and Marvell SATA drivers. As for the rest, I do not think it is critical but it probably does not hurt either.

3. You would not need Marvell's crap if you are only using Intel's SATA ports. If you have one SSD, just connect it to Intel's SATA 6 Gbit port (blue SATA #1). Unfortunately I do not have pictures of the BIOS here and I cannot reboot right now, but the important points are as follows:

- If you are planning to use stock ECC memory, no need to change timings/voltage. But if you are using "overclocker" RAM with XMP, or you wish to overclock , you might need to enter the memory timings manually

- Ensure that the NUMA is ON (chipset configuration -> memory configuration)

- You can also configure the CPU settings for your system role in terms of RAPL (power limits). For example, if you plan to do CPU-hungry stuff, you should configure the CPU for "High Performance Server (HPC)" role, configure the CPU power management to "Performance" and, optionally, maximize the turbo power limits

- You can speed up booting by enabling UEFI-only boot and Windows fast boot - but for this to work, Windows has to be installed in UEFI mode. Same goes for Linux - if you use UEFI boot, your Linux distro has to be installed in UEFI mode.

- There are many more options, but have very specific demands I suggest to live them as-is. You can enable "above 4G decoding" PCI devices to free-up some virtual memory space below 4 GB. If you are using ECC memory, you can also enable both demand and patrol scrubbing in memory configuration to increase the robustness against simple memory errors.

4. No

5. No - but for people that have B0/C0/C1 SNB-EP Xeons slot #3 is actually more stable in terms of boot-detection / PCIe link training.


----------



## AndyE

Quote:


> Originally Posted by *psyq3212*
> 
> - Ensure that the NUMA is ON (chipset configuration -> memory configuration)


For Folding, turn NUMA OFF to gain about 10% performance


----------



## brock2621

Quote:


> Originally Posted by *psyq3212*
> 
> 1. Unless you are using SAS devices, you should use Intel 6 Gbit/s SATA ports (SATA #1-2 blue are 6 Gbit/s). After those two are populated, ti depends on the drivers you'll be using. With Marvell's drivers I am getting decent performance for other SSDs from Marvell 6 Gbit/s SATA ports (gray). However, if you use Windows built-in Marvell driver the results might be worse than using Intel's SATA 3 Gbit/s.
> 
> 2. If you are using Windows, I recommend to update to the latest Intel RSTe (SATA) and Marvell SATA drivers. As for the rest, I do not think it is critical but it probably does not hurt either.
> 
> 3. You would not need Marvell's crap if you are only using Intel's SATA ports. If you have one SSD, just connect it to Intel's SATA 6 Gbit port (blue SATA #1). Unfortunately I do not have pictures of the BIOS here and I cannot reboot right now, but the important points are as follows:
> 
> - If you are planning to use stock ECC memory, no need to change timings/voltage. But if you are using "overclocker" RAM with XMP, or you wish to overclock , you might need to enter the memory timings manually
> 
> - Ensure that the NUMA is ON (chipset configuration -> memory configuration)
> 
> - You can also configure the CPU settings for your system role in terms of RAPL (power limits). For example, if you plan to do CPU-hungry stuff, you should configure the CPU for "High Performance Server (HPC)" role, configure the CPU power management to "Performance" and, optionally, maximize the turbo power limits
> 
> - You can speed up booting by enabling UEFI-only boot and Windows fast boot - but for this to work, Windows has to be installed in UEFI mode. Same goes for Linux - if you use UEFI boot, your Linux distro has to be installed in UEFI mode.
> 
> - There are many more options, but have very specific demands I suggest to live them as-is. You can enable "above 4G decoding" PCI devices to free-up some virtual memory space below 4 GB. If you are using ECC memory, you can also enable both demand and patrol scrubbing in memory configuration to increase the robustness against simple memory errors.
> 
> 4. No
> 
> 5. No - but for people that have B0/C0/C1 SNB-EP Xeons slot #3 is actually more stable in terms of boot-detection / PCIe link training.


Thanks a TON for this info! I really appreciate it. I'll do as instructed. Only question, can you expound on "UEFI" a little? Is it worth doing this? Anyone run into any issues at all? Would you personally recommend it and how much of a difference can it make when already on an SSD?


----------



## TheBlademaster01

Quote:


> Originally Posted by *psyq3212*
> 
> Hi TheBlademaster01,
> 
> Can you run Windows? If you do not have Windows installation, you can just copy the contents on the Windows installation ISO to a USB stick and boot into the Windows setup, then go to "recover my computer" option.
> 
> Also, copy the contents of HWiNFO installation ZIP file (http://www.hwinfo.com/)
> 
> Then, after you boot Windows (or Windows setup -> "Repair my computer" -> Command Prompt", launch the HWiNFO.
> 
> Then go to memory summary - it will display the memory configuration (single / dual / quad channel)
> 
> Please let me know if it helped.


It reports dual channel. My performance is not on par with other systems though. RAM looks okay after further inspection

Look at this

http://browser.primatelabs.com/geekbench2/2150345

http://browser.primatelabs.com/geekbench2/2151584

And mine...

http://browser.primatelabs.com/geekbench2/2190040


----------



## AndyE

Is your BIOS set to memory mirroring?


----------



## TheBlademaster01

No, I haven't changed much in the BIOS except for that I just now changed channel interleaving to 2 and disabled NUMA.


----------



## seross69

good luck


----------



## TheBlademaster01

Andy, could you change multi to 24x 104MHz and run GeekBench and/or CineBench?


----------



## TheBlademaster01

What I also noticed btw is that the board loops 3 times before it boots into something


----------



## psyq3212

@TheBlademaster01,

Hmm... these are things I would check:

- What are your memory timings? Can you check that the memory timing is detected properly from SPD - if not, you might need to manually enter them
- You can try disabling QPI l0 and l1 power states in the BIOS, if you do that, QPI links will stay on the highest power state and not throttle down
- Try experimenting with NUMA, if disabling NUMA in BIOS brings you better results, this is because the given application is not NUMA-optimized

@brock2621,

UEFI brings some advantages like slightly faster booting - but if you already have OS(es) installed, I do not think it warrants a OS-reinstall. Basically, all new Intel platforms are UEFI, and they load additional compatibility layer called CSM for "emulating" BIOS - if CSM is enabled, the operating system still thinks it runs on BIOS machine. If your OS is installed as UEFI, this means you do not need BIOS compatibility layers and they can be disabled.


----------



## brock2621

Quote:


> Originally Posted by *psyq3212*
> 
> @brock2621,
> 
> UEFI brings some advantages like slightly faster booting - but if you already have OS(es) installed, I do not think it warrants a OS-reinstall. Basically, all new Intel platforms are UEFI, and they load additional compatibility layer called CSM for "emulating" BIOS - if CSM is enabled, the operating system still thinks it runs on BIOS machine. If your OS is installed as UEFI, this means you do not need BIOS compatibility layers and they can be disabled.


I guess there's not any tutorial videos anyone made by chance that walks me through this process for this particular board? Either way thanks so much for the help!


----------



## TheBlademaster01

There are no states in the bios. Only fast/slow and adaptive/performance IIRC.

Bios is at 0503 btw. Do I need to flash it? And what is the best way. Have nightmares of flashing my MSI boards


----------



## psyq3212

Put it @"Performance".

No need to update the BIOS unless you want to run Windows Server 2012 / Windows 8, since pre-cap (3xxx) BIOS-es were not compatible.

You should really check the memory timings and frequency - it can happen that board picked worse timings than what the memory is rated for.


----------



## TheBlademaster01

Okay, I don't know if you've run GeekBench an CineBench before but even with 8GB of RAM my floating point performance is nearly the same...

I get 18.60 with single channel and highest was 18.74 in Kubuntu w/ WINE. GeekBench is around 24.5k









Floating point only 44k.


----------



## AndyE

Quote:


> Originally Posted by *TheBlademaster01*
> 
> Andy, could you change multi to 24x 104MHz and run GeekBench and/or CineBench?


Sorry,
I am travelling and don't have access to the system right now.

Have you checked the stream benchmark ?
http://www.cs.virginia.edu/stream/

This tests the memory subsystem.
With 1600 MHz ECC (CL11) Ram my system achieved 78 GB/sec with this m/b, out of 102.4 GB/s theoretical bandwidth. Each channel has 12.8 GB/s (theor. max). Times 8 channels = 102.4 GB/s
75-80% of th. max is usually ok.

If your system max out at 40 GB/s, you might run only on 2 channels per socket.

Andy


----------



## TheBlademaster01

Quote:


> Originally Posted by *AndyE*
> 
> Sorry,
> I am travelling and don't have access to the system right now.
> 
> Have you checked the stream benchmark ?
> http://www.cs.virginia.edu/stream/
> 
> This tests the memory subsystem.
> With 1600 MHz ECC (CL11) Ram my system achieved 78 GB/sec with this m/b, out of 102.4 GB/s theoretical bandwidth. Each channel has 12.8 GB/s (theor. max). Times 8 channels = 102.4 GB/s
> 75-80% of th. max is usually ok.
> 
> If your system max out at 40 GB/s, you might run only on 2 channels per socket.
> 
> Andy


Hmm, I can't seem to get that running in Ubuntu. I'll get my Windows distro in a couple of days.


----------



## brock2621

So I just installed 8x8gb crucial ballistix with supposed timings of 8-8-8-24 at 1600 and it's booting at 1333 :/.

Do I just change the speed in bios? Do I need to change voltage as well? These currently run at 1.35 I believe.


----------



## anubis1127

Quote:


> Originally Posted by *brock2621*
> 
> So I just installed 8x8gb crucial ballistix with supposed timings of 8-8-8-24 at 1600 and it's booting at 1333 :/.
> 
> Do I just change the speed in bios? Do I need to change voltage as well? These currently run at 1.35 I believe.


I've always had to manually specify the timings, and voltage for RAM on this board. I just put in 8x4gb g.skill last night that is also 1600 8-8-8-24, but I left the Command Rate on Auto, and the board chose 1n, which ended up being severely unstable. I manually set it to 2n, which is what the g.skill site lists for this RAM kit, and today things have been much better.


----------



## brock2621

Yeah I think imma have to do it manually. The only problem I foresee is that I can't find any info on the command rate for my kits. I bought 2x of the BLT4K8G3D1608ET3LX0 (32gb) kits and i can't find anything on the command rate :/

Can I manually tell it to run in quad channel or is that left up to the board?


----------



## AndyE

Quote:


> Originally Posted by *brock2621*
> 
> So I just installed 8x8gb crucial ballistix with supposed timings of 8-8-8-24 at 1600 and it's booting at 1333 :/.


I have 2x32GB crucial dominator platinum kits as well, which can be O/C up to 2133 MHz, yet the standard SPD is set to 13333MHz. Just change it manually in the BIOS.
The # of channels supported is primarily driven by you placing the DIMMs in the proper slots. The BIOS will optimize based on physical arrangement.


----------



## brock2621

Quote:


> Originally Posted by *AndyE*
> 
> I have 2x32GB crucial dominator platinum kits as well, which can be O/C up to 2133 MHz, yet the standard SPD is set to 13333MHz. Just change it manually in the BIOS.
> The # of channels supported is primarily driven by you placing the DIMMs in the proper slots. The BIOS will optimize based on physical arrangement.


EDIT: Should I enter exact same settings for secondary timings?

Does it matter the order if I ordered 2 of the same kits just like you since they are all the same?


----------



## brock2621

Ok, so I'm get "intel boot" flashing twice in a row and then finally booting into windows. Any quick fixes from anyone? Is it failing to boot twice in a row before finally booting the 3rd time?

And my geekbench score seems really low? http://browser.primatelabs.com/geekbench2/2196424


----------



## brock2621

Quote:


> Originally Posted by *TheBlademaster01*
> 
> What I also noticed btw is that the board loops 3 times before it boots into something


My board does the same thing. It's driving me nuts! Were you able to figure out how to keep it from looping 3 times?


----------



## TheBlademaster01

No, I didn't find a solution









About GeekBench same here. Also can't figure out why it's low. Though your score doesn't seem that bad franted I don't know what 2687W should score.
i've got 2665s @2.5 and get this

http://browser.primatelabs.com/geekbench2/2186672

I've tried to disable EIST, NUMA, revert to standard settings but no luck.

I know 2687W should get about 24-25 points in CineBench.


----------



## brock2621

Quote:


> Originally Posted by *TheBlademaster01*
> 
> What I also noticed btw is that the board loops 3 times before it boots into something


Ok figured these out! Do the following steps and it will solve all the boot loops:

Turn off all the marvell controllers

Turn boot seeking to normal instead of endless PXE

Turning PXE functions off for the onboard LAN controllers


----------



## phillydee

Quote:


> Originally Posted by *brock2621*
> 
> Ok figured these out! Do the following steps and it will solve all the boot loops:
> 
> Turn off all the marvell controllers
> 
> Turn boot seeking to normal instead of endless PXE
> 
> Turning PXE functions off for the onboard LAN controllers


Haaa!!! OMG this will be great--I wasn't sure if this WAS the PXE functions... I'll do this next cold boot! Woo!


----------



## TheBlademaster01

Wow, thanks man. That solved it. I'm working on performance still


----------



## brock2621

EDIT


----------



## TheBlademaster01

E5-2665 @2.3 w/ 1333MHz Dual Channel

http://browser.primatelabs.com/geekbench2/2197860

E5-2665 @2.5 w/ 800MHz Dual Channel

http://browser.primatelabs.com/geekbench2/2197872

E5-2665 @2.5 w/ 1600MHz Dual Channel

http://browser.primatelabs.com/geekbench2/2197898

I really don't know at this point...


----------



## brock2621

Quote:


> Originally Posted by *TheBlademaster01*
> 
> E5-2665 @2.3 w/ 1333MHz Dual Channel
> 
> http://browser.primatelabs.com/geekbench2/2197860
> 
> E5-2665 @2.5 w/ 800MHz Dual Channel
> 
> http://browser.primatelabs.com/geekbench2/2197872
> 
> E5-2665 @2.5 w/ 1600MHz Dual Channel
> 
> http://browser.primatelabs.com/geekbench2/2197898
> 
> I really don't know at this point...


What does CPU-z say about your ram?


----------



## TheBlademaster01

I'm in Ubuntu, but I've confirmed that it's dual channel.

I'm thinking about a BIOS flash but I have B0 chips. Any advice gents?

I see this a lot. Same hardware, totally different scores

Dell T7600 w/ dual E5-2665s

*(high)*

http://browser.primatelabs.com/geekbench2/2127617

http://browser.primatelabs.com/geekbench2/2127613

http://browser.primatelabs.com/geekbench2/2127604

http://browser.primatelabs.com/geekbench2/2127613

*(low)*

http://browser.primatelabs.com/geekbench2/1507398

http://browser.primatelabs.com/geekbench2/1507344


----------



## TheBlademaster01

Here's a system with same spec as mine but quad channel. Same BIOS and even Windows but a lot higher score than mine...

http://browser.primatelabs.com/geekbench2/1558516


----------



## anubis1127

Quote:


> Originally Posted by *TheBlademaster01*
> 
> I'm in Ubuntu, but I've confirmed that it's dual channel.
> 
> I'm thinking about a BIOS flash but I have B0 chips. Any advice gents?
> 
> I see this a lot. Same hardware, totally different scores
> 
> Dell T7600 w/ dual E5-2665s
> 
> *(high)*
> 
> http://browser.primatelabs.com/geekbench2/2127617
> 
> http://browser.primatelabs.com/geekbench2/2127613
> 
> http://browser.primatelabs.com/geekbench2/2127604
> 
> http://browser.primatelabs.com/geekbench2/2127613
> 
> *(low)*
> 
> http://browser.primatelabs.com/geekbench2/1507398
> 
> http://browser.primatelabs.com/geekbench2/1507344


Those are likely retail chips though, which their turbo would be around ~300mhz faster than your chips OC'd.

I can do some testing this weekend. I should be able to set my chips to 2.5ghz, and let you know what I get.


----------



## TheBlademaster01

Quote:


> Originally Posted by *anubis1127*
> 
> Those are likely retail chips though, which their turbo would be around ~300mhz faster than your chips OC'd.
> 
> I can do some testing this weekend. I should be able to set my chips to 2.5ghz, and let you know what I get.


Cool









Yeah, I know but even ikem's 2650 ES and other 1999MHz ES chips are beating mine









18.6 in CineBench and 24.5k in GeekBench is quite low for 2P.

Considering 3930K at 5GHz can do 15 in CineBench.

Edit:

I also beat some of those retail 2665s. That's what I don't get


----------



## brock2621

Ugh... currently my EVGA Titan is NOT being recognized by Nvidia's site, or the GeForce Experience app. Anyone else having this issue? Anyone else own a Titan?


----------



## lloyd mcclendon

For a few weeks now I've been noticing this very high pitched noise coming from my machine. I figured it was the water pumps or a fan. Last night it was driving me crazy, it's like a dog whistle and if you listen in on it for a few seconds, it has a terrible "brain melting" effect. So, I stopped the pumps and the fans, but it continued, stuck my head in there and to my surprise and confusion it was coming from right around the CPUs.

In searching around about this I did find a few others with the same issue. Apparently it is (suspect?) voltage regulators and/or capacitors. Others also said it can happen if you enable certain C states in the bios. Well, a few weeks ago I had turned on all of the C0/6/7 states. I've restored that to the defaults and the noise is gone.









Any thoughts? To be honest I really don't even know what C states are.

In other news, I again got the stones to try getting up to 105mhz and 2133 again, and it "just worked" with v5004 - seems stable so far.


----------



## psyq3212

Noise in certain C-states is haunting certain Intel CPU-s across multiple generations.

Frankly speaking, I thought this problem is gone - but obviously it is not.

Try changing "Clock Spread Spectrum" setting (if it is ON, turn it OFF and vice versa). Spread spectrum setting will alter the clocks just a little bit to avoid interference. It might help solve the whining/buzzing issue.


----------



## TheBlademaster01

Here's my HWinfo screenie



Performance in Windows is even lower









http://browser.primatelabs.com/geekbench2/2206862


----------



## AndyE

Quote:


> Originally Posted by *TheBlademaster01*
> 
> Performance in Windows is even lower
> 
> 
> 
> 
> 
> 
> 
> 
> http://browser.primatelabs.com/geekbench2/2206862


That the Windows Geekbench score is lower than the Linux is not an issue - might depend on compiler optimizations, etc. Compare only within the same category.
Quote:


> Originally Posted by *TheBlademaster01*
> 
> Here's a system with same spec as mine but quad channel. Same BIOS and even Windows but a lot higher score than mine...
> 
> http://browser.primatelabs.com/geekbench2/1558516


You should be more careful when you compare numbers: The link you provided shows Geekbench results for the 64bit version, not like your 32bit version.

Your numbers look for the CPU normal (Arithmethic, Floating, Memory) the only anomaly is the Stream Benchmark value. It is about 2/3 of where it should be - the impact of your dual channel setup.

Why haven't you updated the BIOS?
HWInfo shows 0503 - there were 6 BIOSes since then

I'd suggest that you double and triple check your BIOS setup (memory section).
Another potential issue could be a faulty DIMM which triggers your BIOS into dual channel mode
Did you program the speed values into your DIMMs or are they factory?


----------



## TheBlademaster01

I have B0 chips, .cap doesn't work I believe.

I do have dual channel (only 4 modules installed)

Does 64-bit score higher than 32-bit?

And I get 21k/44k on the CPU in GeekBench. 18.6 in linux CineBench and 18.2 in Windows. There are ES 2650s scoring better than mine. I'm pretty sure it's not what it should be


----------



## TheBlademaster01

For reference, this is what my 2600K does in Geekbench

http://browser.primatelabs.com/geekbench2/2177304


----------



## AndyE

Quote:


> Originally Posted by *TheBlademaster01*
> 
> I have B0 chips, .cap doesn't work I believe.
> 
> I do have dual channel (only 4 modules installed)
> 
> Does 64-bit score higher than 32-bit?
> 
> And I get 21k/44k on the CPU in GeekBench. 18.6 in linux CineBench and 18.2 in Windows. There are ES 2650s scoring better than mine. I'm pretty sure it's not what it should be


1) they do
2) If you have four DIMMS installed and 2 CPU sockets filled, then it is natural that you have only 2 channel ops. For 4-channel operation with a dual Xeon system, you need 8 DIMMS
3) yes
4) Your B0 version can't be compared to retail 2650 versions. The EIST/Turbo is more limited.
Quote:


> For reference, this is what my 2600K does in Geekbench


Would be irrelevant for me.
Too many well known benchmarks are frequency dependent, which worked well as long as speed increases came with higher frequencies and better ILP.
Going forward, those older benchmarks will loose even more relevancy (until they are rewritten to explicitly use manycore architectures effectively). It is a limitation by the benchmark, not the hardware. Take NUMA as an example.


----------



## TheBlademaster01

Quote:


> Originally Posted by *AndyE*
> 
> 1) they do
> 2) If you have four DIMMS installed and 2 CPU sockets filled, then it is natural that you have only 2 channel ops. For 4-channel operation with a dual Xeon system, you need 8 DIMMS
> 3) yes
> 4) Your B0 version can't be compared to retail 2650 versions. The EIST/Turbo is more limited.
> Would be irrelevant for me.
> Too many well known benchmarks are frequency dependent, which worked well as long as speed increases came with higher frequencies and better ILP.
> Going forward, those older benchmarks will loose even more relevancy (until they are rewritten to explicitly use manycore architectures effectively). It is a limitation by the benchmark, not the hardware. Take NUMA as an example.


Okay, thanks for the response









My main problem initially was whether or not I have dual channel vs single channel but that problem was already solved. I knew I wasn't running quad









I actually compared them to ES 2650s and other ES 2665s, 2670s and 2660s. Actually, a 3930K OC'ed to 5GHz comes very close to my 2.5GHz Xeons (15p vs 18p). I really have a hard time believing that this performance is okay for 2.5GHz E5s but I can't exactly find a problem either other than not having quad channel. Do you think that this is bottlenecking my setup?


----------



## AndyE

So you have 4 DIMMS, not 8?

If yes, you take the single most important differentiator to i7-39xx systems away. Memory bandwidth

If your applications need it or not, is a different story, but #1 difference is mem bandwidth.

The second problem you are facing is the application layer.
As long as you use either for real-use or benchmarking "inappropriate" software, the benefit of a dual Xeon system won't be visible. (It is similar to Unigine Valley, which is supposed to benchmark your GPU. Starting at a certain GPU performance level, the biggest impact on the score is the frequency of the CPU, not the capabilities of the GPU. It is bascially a software induced issue)

Many games haven't even mastered multicore architectures to the fullest extend, let alone came close to tackle NUMA architectures.

There are basically 2 dimensions to optimize computer systems for:
1) For minimum latency (reaction time)
2) for maximum throughput
The architectures for both are different
Your 2600K system is a representative of (1), the dual Xeon system of (2)

For instance:
Your dual Xeon system will deliver 200-250k points for FAH folding, your 2600K will max out in the 30k points range.

Your DualXeon system will most likely suck for any of the current games on the market. This is not a HW issue per se, it is the limitation of the software architectures of contemporary games, which leaves many of the increased capabilities on the sidelines.

Back to your system:
If you run your dual Xeon on dual channel, the memory bandwidth of your Xeons is approx the same as for an i7-39xx system. Add to this configuration the higher frequency of the i7 and the inability of most benchmark software to use all cores effectively, its no wonder your benchmark score is low.

Use the system with appropriate software and it will be a very effective companion (with 8 DIMMs).


----------



## TheBlademaster01

However, I will be using this rig for 3D modelling and rendering. It scales linear with cores and clocks (most engines I use that is) since workload is fairly parallel. I was pretty sure I should get around 2x the points of a 2600K system in that. CineBench takes advantage of the cores but for some reason my score is on par with Nehalem 2P and a 2GHz ES2 Sandy EP chip.

It could be BIOS, BIOS settings or the RAM as far as I can judge.

I thought part of GeekBench also tested multi core performance.

I'm not even sure I could get over 200k on a P8102 with the theoretical performance it is showing now (seeing QRB is non-linear). I really need some comparison material though.


----------



## TheBlademaster01

Sorry for keeping double posting but my iPad doesn't like editing









Here's a 2.2GHz ES CPU beating it by about 13%

http://www.overclock.net/t/1099447/coolaler-jc-dual-socket-lga-2011-xeon-cinebench-scores/0_30


----------



## AndyE

You haven't answered the question: Are you running the system with 4 memory sticks?
If yes: Add another 4 to get the performance these CPUs are capable of.
You are consciously degrading the system performance for large memory applications.

Would you run your 2600K with a single memory stick?


----------



## TheBlademaster01

I have only 4 as mentioned earlier. I have 4 extra ones enroute tomorrow prolly, at least this week







. The thing is, I have never tested my 2600K with only 1 stick and really wished I did because I'm not sure whether it will actually solve the problem.

I appreciate the time you're putting in to this though. It's not that I'm unhappy about the performance but hearing all these stories about the board I'm wondering whether I have another issue that is limiting things. I like this system if only for the efficiency (consumes less power than my 2600K one).


----------



## AndyE

thanks & no prob.

It is a matter of balanced system design to allow the CPUs to be feeded by fast memory.
At the end of the day it is dependent on the architecture of the application how efficient it leverages the system architecture.
Be aware that applications like cinebench might no show an improvement when you plug in your 4 sticks. Most benchmark utilities tend to have a rather small dataset (covered well by the caches).

wrt to the 18.6 Cinebench score and the 13% better result of "the other guy". You don't know what has been modified by this guy, is the system stable only for the duration of the benchmark, or is everything at stock? What tweaks are applied in the BIOS? Is the clock timing original? Unless all this contextual information is available, a comparison is hard to make.

Get to a decent all up performance, stop benchmarking and start using it for everyday work - you won't recognize anyway 10% perf difference.

BTW, 18.6 seems to be ok, as the same CPU with 1600 MHz ECC sticks (11-11-11-27) will do 18.0.

If you want to see the impact on memory performance use Intel's Linpack benchmark. (take the Linux version, it is better optimized than the Windows version)
Linpack can be configured to use all of your memory, which is different to most other benchmark utilities.


----------



## psyq3212

You are basically losing 25.6 * 2 GB/s = 51.2 GB/s of memory bandwidth, which is huge (assuming your memory runs @1600 MHz).

Theoretical maximum memory bandwidth of your system with 8 DIMMs populated will be 102.4 GB/s, and you are currently using only half of that.

This is almost certainly going to improve your benchmark results.


----------



## psyq3212

Quote:


> Originally Posted by *TheBlademaster01*
> 
> I have B0 chips, .cap doesn't work I believe.
> 
> I do have dual channel (only 4 modules installed)
> 
> Does 64-bit score higher than 32-bit?
> 
> And I get 21k/44k on the CPU in GeekBench. 18.6 in linux CineBench and 18.2 in Windows. There are ES 2650s scoring better than mine. I'm pretty sure it's not what it should be


Latest BIOS that works with your SNB-EP B0 is 3506.

Although I really doubt BIOS upgrade will do anything for the benchmark results (although ASUS switched to NUMA-on by default in one of the first updates AFAIR)

Bios #5004 is breaking compatibility with B0 SNB-EP, because ASUS switched to IvyTown MRC (reference code), which does not work with ES2 SNB-EPs (I think Intel dropped support long time ago, but ASUS did not update the MRC until v5xxx). Assuming Intel won't come with further MRC upgrades after August this means that SNB-EP ES2 compatibility will not be coming back, so make sure if you plan to upgrade BIOS not to go further than ver. 3506


----------



## TheBlademaster01

Quote:


> Originally Posted by *psyq3212*
> 
> Latest BIOS that works with your SNB-EP B0 is 3506.
> 
> Although I really doubt BIOS upgrade will do anything for the benchmark results (although ASUS switched to NUMA-on by default in one of the first updates AFAIR)
> 
> Bios #5004 is breaking compatibility with B0 SNB-EP, because ASUS switched to IvyTown MRC (reference code), which does not work with ES2 SNB-EPs (I think Intel dropped support long time ago, but ASUS did not update the MRC until v5xxx). Assuming Intel won't come with further MRC upgrades after August this means that SNB-EP ES2 compatibility will not be coming back, so make sure if you plan to upgrade BIOS not to go further than ver. 3506


Awesome, thanks









Yeah, I'm thinking of buying a new BIOS chip when upgrading to Ivy and get a later revision this time








Quote:


> Originally Posted by *AndyE*
> 
> thanks & no prob.
> 
> It is a matter of balanced system design to allow the CPUs to be feeded by fast memory.
> At the end of the day it is dependent on the architecture of the application how efficient it leverages the system architecture.
> Be aware that applications like cinebench might no show an improvement when you plug in your 4 sticks. Most benchmark utilities tend to have a rather small dataset (covered well by the caches).
> 
> wrt to the 18.6 Cinebench score and the 13% better result of "the other guy". You don't know what has been modified by this guy, is the system stable only for the duration of the benchmark, or is everything at stock? What tweaks are applied in the BIOS? Is the clock timing original? Unless all this contextual information is available, a comparison is hard to make.
> 
> Get to a decent all up performance, stop benchmarking and start using it for everyday work - you won't recognize anyway 10% perf difference.
> 
> BTW, 18.6 seems to be ok, as the same CPU with 1600 MHz ECC sticks (11-11-11-27) will do 18.0.
> 
> If you want to see the impact on memory performance use Intel's Linpack benchmark. (take the Linux version, it is better optimized than the Windows version)
> Linpack can be configured to use all of your memory, which is different to most other benchmark utilities.


Again, thanks a lot man








Quote:


> Originally Posted by *psyq3212*
> 
> You are basically losing 25.6 * 2 GB/s = 51.2 GB/s of memory bandwidth, which is huge (assuming your memory runs @1600 MHz).
> 
> Theoretical maximum memory bandwidth of your system with 8 DIMMs populated will be 102.4 GB/s, and you are currently using only half of that.
> 
> This is almost certainly going to improve your benchmark results.


How did you calculate that?


----------



## psyq3212

DDR3 1600 MHz gives you 1600 MT/s which multiplied with 8 = 12800 MB/s

Now, you have 4 channels on each CPU, so you multiply this with 4, and you get 51200 MB/s

Also, you have 2 CPUs, each of then can attain the above bandwidth, so multiplied by 2 = 102400 MB/s


----------



## AndyE

Quote:


> Originally Posted by *psyq3212*
> 
> Also, you have 2 CPUs, each of then can attain the above bandwidth, so multiplied by 2 = 102400 MB/s


To add on this to the question of the OP:
max "real world performance" with the Stream benchmark will be around 80 GB/sec.
All other conditions will be lower due to:
1) address misalignment
2) suboptimal page boundaries
3) suboptimal rank interleave
4) DIMM contention
5) TLB misses
6) etc, etc, ...


----------



## squidman

Ok about to purchase damn mb, due to open box (2600 nok vs 3700 nok or 437 usd vs 623 usd here in Norway/fargoland)....but how safe is open-box? I've sent an email of course to resellers, but iff I'm wrong and this board is a doa return, I'm still out of pocket 450 buckaroos. The newegg customer reviews are not exactly encouraging...something like half are lemons.

Guess will pick up just one low end e5-2620 2ghz for now, the ef-v2 are coming out in september....wish the didn't have to be matched (or do they?). Hate buying last year's tech (this board is quite old, but still the most extreme as of writing), and again what a difference from a first-gen i7 lappie!

So...should I wait (this is mostly to get rendering times to speed up) or should I wait for some hero to compile e.g. for Vue Xstream, Arnold renderVray etc for Xeon Phi boards (whatever happened to them?). Or do a helmer? (http://helmer.sfe.se/).

I have maybe one-two days to pick up the open box asus board @ 450 usd, hope it's not doa...or I can buy full price for 650 usd...

cookie out...been renovating, wood-chopping and other norwegain non-nerdy things! (but yes we rented giant-ass wood-chopping machines...! my coz is an engineer! goddamn, when you can get a giant-ass wood-chopping machine to do it!)...


----------



## brock2621

Making progress! Need to do some wire/cable management but it's up and running!


----------



## TheBlademaster01

Quote:


> Originally Posted by *squidman*
> 
> Ok about to purchase damn mb, due to open box (2600 nok vs 3700 nok or 437 usd vs 623 usd here in Norway/fargoland)....but how safe is open-box? I've sent an email of course to resellers, but iff I'm wrong and this board is a doa return, I'm still out of pocket 450 buckaroos. The newegg customer reviews are not exactly encouraging...something like half are lemons.
> 
> Guess will pick up just one low end e5-2620 2ghz for now, the ef-v2 are coming out in september....wish the didn't have to be matched (or do they?). Hate buying last year's tech (this board is quite old, but still the most extreme as of writing), and again what a difference from a first-gen i7 lappie!
> 
> So...should I wait (this is mostly to get rendering times to speed up) or should I wait for some hero to compile e.g. for Vue Xstream, Arnold renderVray etc for Xeon Phi boards (whatever happened to them?). Or do a helmer? (http://helmer.sfe.se/).
> 
> I have maybe one-two days to pick up the open box asus board @ 450 usd, hope it's not doa...or I can buy full price for 650 usd...
> 
> cookie out...been renovating, wood-chopping and other norwegain non-nerdy things! (but yes we rented giant-ass wood-chopping machines...! my coz is an engineer! goddamn, when you can get a giant-ass wood-chopping machine to do it!)...


I bought used and made sure it was working. I would be concerned about buying open box.

@brock

Looking good


----------



## squidman

Well the open box is on its way and covered by warranty here...it was opened at the webshop/warehouse to make sure nothing is missing. Great deal! Then even better, found (AFTER I bought the e5-2620 of course) a couple of china-sourced QA8X for 600 bucks including shipping







Now of course they have to work, not be confiscated by customs etc, and I'll probably have to pay 25% VAT on top....but here they go for 22000 nok/3714 usd a pair, pretty much the cost of my entire build!









Mobo arriving today, don't suppose there's anyway of determining the BIOS version from the cmos chip or the packaging? Will probably pick up some programmed extra chips with various earlier bios versions, can be had for 8-9 euros. Anyone with a stable build and qa8x's? What bios version?

Rest will come in drips and drabs..

Obsidian 900d (need to tap holes, remove some standoffs)
2xWD 3TB Green
2x Samsung 840 250GB ssd in raid 0 for OS and 3d programs
2x el cheapo kingston 64gb ssd for caching
(4x8gb) CORSAIR 32GB DDR3 1866 Q/KIT VENGEANCE
2x corsair h80i
corsair 1200 ax gold (hope this works, some peps have had success, others not)
zotac 770 gtx 4gb (basically a 680/keplar 104, hope it works)

Total build cost will be around 4000 usd iff I can flog/return the xeon 2620 - super stoked!









Oh yeah...what are the chances a customs officer recognizes an ES chip? With 'intel conf...' stamped on it? Haven't heard of a buyer yet being penalized, but you never know. And please no lectures, intel makes enough moola...I don't feel sorry for monopolists..
What a change from a lowly albeit i7 laptop!


----------



## squidman

Quote:


> Originally Posted by *psyq3212*
> 
> Latest BIOS that works with your SNB-EP B0 is 3506.
> 
> Although I really doubt BIOS upgrade will do anything for the benchmark results (although ASUS switched to NUMA-on by default in one of the first updates AFAIR)
> 
> Bios #5004 is breaking compatibility with B0 SNB-EP, because ASUS switched to IvyTown MRC (reference code), which does not work with ES2 SNB-EPs (I think Intel dropped support long time ago, but ASUS did not update the MRC until v5xxx). Assuming Intel won't come with further MRC upgrades after August this means that SNB-EP ES2 compatibility will not be coming back, so make sure if you plan to upgrade BIOS not to go further than ver. 3506


OK questions answered...thanks for that! Hope 3506 addresses compatibility issues with gk104...otherwise it will be a different graphics card.

One more question...have heard that unused mobo standoffs can cause shorts...should or shouldn't they be removed? Still waiting on the case d900, not sure if standoffs are screwed in or permanent...assume are screwed in as needed. Anyone tried tapping holes in eatx trays for this mobo, and if so what size tap and hole?


----------



## ZDngrfld

Quote:


> Originally Posted by *squidman*
> 
> OK questions answered...thanks for that! Hope 3506 addresses compatibility issues with gk104...otherwise it will be a different graphics card.
> 
> One more question...have heard that unused mobo standoffs can cause shorts...should or shouldn't they be removed? Still waiting on the case d900, not sure if standoffs are screwed in or permanent...assume are screwed in as needed. Anyone tried tapping holes in eatx trays for this mobo, and if so what size tap and hole?


You should be able to use the majority of the standard mounting holes. I think there's two, maybe three that don't line up since it is a SSI-EEB board. Just get some of those plastic standoffs to fill those.


----------



## squidman

Ok yet another question.....regarding build. How do you enable ssd caching via the marvell/asus sata ports? I guess I'm planning on installing win7 64 ultimate via usb-stick. But what is the procedure as far as what goes first?

For reference, I want 2x250gb ssd's in raid 0 in the topmost blue c602 sataIII ports, and then 2 WD Green hdd with 2 small ssd's in the grey marvell sata ports ( for ssd caching), and then basically populate the rest jbod with my existing external drives (all 14 TB of them or something! do a lot of video/3d etc!)...

For that matter, what is the best file system for any/all of the above? NTFS? Exfat? etc. Does the Asus smart-caching allow TRIM? Ditto the c602 sata III ports (blue at the top), is trim allowed in raid 0?

How did you guys do it, in step by step manner? Sorry, first workstation build, first time installing from usb, first time with dual cpu's etc. Will the bios even start iff none of the drives are formatted?

Thanks again, the newb...really appreciate this thread, only place with solid info on this mobo...









Edit: chicken and egg question....the ssd asus smart-caching is via their windows utility, so I guess I have to install win7 first! That would be on the raided 0 ssd's...but I assume you set up raid etc. on booting up via the bios? Then you restart once micro$oft has installed win7, and start installing all the drivers. Then iff you've made thus far without bsod or error codes you can use the asus smart-caching utility to 'connect' the ssd's with the hdd's on the marvell (pos! yes I know people are critical!). Does that sound about right?

Or do you set up the drives first in BIOS?, and THEN install win7 etc? Asus documentation sucks ball$

cookie out..


----------



## squidman

Yet another question on build from scratch: regarding safest way to install components...

What is best...take out mobo tray (iff possible on 900d), install as much as possible, place back in case, or place case on floor with tray nearest ground and install? Have yet to see a build step for step on youtube etc. regarding corsair 900d.

How are the backplate thingies going to work on 900d? don't we need 2? for dual xeons? 2 holes for cpu? srry still newb here, venturing fearlessly into b0 stepping territory


----------



## AndyE

Don't overload the DMI connection with all your SSD/HDs connected to SATA ports on the motherboard. The total sum of transfers of USB ports, SATA ports and LAN ports is approx 1.5 - 1.6 GB/s. For higher speed I/O use a relatively cheap PCI express host bus adapter. I am using LSI 9207-8i's for about 300 USD. Good for up to 4.2 GB/s transfer rates per card.

TRIM:
Only needed for SSDs which don't do real time garbage collection. Usually raid systems don't support TRIM functionality, exception being the Intel chipset.

Build order:
Both is possible. My default approch is simple. Put in the mobo first, then put in all the stuff onto the m/b - CPU, RAM, cables, ...

Filesystem:
If you care about your data - NTFS, much more stable.


----------



## BlackenedTush

Hey Squidman,
*
Deciding whether to populate the board inside or outside*...
A lot of consideration has to do with that initial post test. How much time and effort will you have invested to test for post. If it doesn't work you'll have wasted it. So if you are in a crunch for time, a conservative approach would estimate and compare how much time and work is involved in getting it post-ready both in the case & outside the case.

Like you, my first complete build was with this board. My default approach was the same as AndyE's... I put the big Z9PE-D8 in the case first and built from there. But when I had my initial problems posting, getting RAM to be detected, and getting the GPU to send a DVI video signal to the monitor, I had already gone thru a decent amount of work fitting the mobo into my tight HAF X case. So at that point I was very reluctant to even consider RMAing the board. That was because of the work I had already invested and how much would be involved to get it back where it was. Considering my skill level and confidence at the time, I considered redoing it all to be a pretty big deal. It turns out that I didn't need to RMA the board after all; but even so, my reluctance affected my approach to diagnosis. It turned out that the mobo was not "RMA material"; bu if it had been, I would have had a lot to undo and then a lot to redo.

Since then I had looked at the *



* where Paul suggested doing the initial assembly/testing of the board _outside of the case_ (sitting on it's own packaging). That approach is designed to achieve that critical first successful post with minimal investment. On my second build I took that advice. It did seem like a wiser, more methodical approach. And actually, at no point was I ever worrying, "what if it doesn't work?".

For you, it's probably the same amount of work either way. Yes it's easier to get at everything out in the open like Newegg Paul demonstrates. And you don't need to screw in the board, but for you there are those Corsair H2O-based coolers with their unwieldy hoses and radiators... Once you've attached those beasts and scored a successful post, I could see how it might be harder to manage putting the mobo into the case with those floppy things attached (unless you have a couple of extra hands). Plus your case is so large open to begin with... _plus_ it has ready-made accommodations for those rads already. So it's probably easiest to use the case.

*As for the standoffs*...
Just make sure there are no metal standoffs touching your board anywhere that there isn't a mounting screw hole. I have a Corsair Vengeance case for my second build. That one and some others I've seen have one standoff in the middle which is a plastic pin. It helps by quickly orienting your board when you're putting the ports through the back plate. That's kinda nice.

Plastic standoffs were suggested by ZDngrfld. A very good suggestion. You'll most likely need them for two holes as I said in an earlier post. If you do decide to get 2 plastic standoffs for that, you will need to get the kind that doesn't have the threaded screw on one end (or you'll need to cut the threaded screw part off) because that type is designed to screw into a threaded hole in your case and of course there won't be a hole there. My approach was to file the threads off of 2 brass standoffs and screw them to the motherboard... Plastic SOs simply snap into the motherboard. But you will definitely need some kind of support for those two holes because both are centered in large spans of PCB where there aren't any any other standoffs for additional support. And without support there, the act of pushing RAM sticks in could impose extreme flex on the board

BT


----------



## squidman

Quote:


> Originally Posted by *AndyE*
> 
> Don't overload the DMI connection with all your SSD/HDs connected to SATA ports on the motherboard. The total sum of transfers of USB ports, SATA ports and LAN ports is approx 1.5 - 1.6 GB/s. For higher speed I/O use a relatively cheap PCI express host bus adapter. I am using LSI 9207-8i's for about 300 USD. Good for up to 4.2 GB/s transfer rates per card.
> 
> TRIM:
> Only needed for SSDs which don't do real time garbage collection. Usually raid systems don't support TRIM functionality, exception being the Intel chipset.
> 
> Build order:
> Both is possible. My default approch is simple. Put in the mobo first, then put in all the stuff onto the m/b - CPU, RAM, cables, ...
> 
> Filesystem:
> If you care about your data - NTFS, much more stable.


Thanks both of you for the help! Really appreciated...

File systems...how about exfat, ex2, ex3 etc? Seemed to recall using those on my NAS's (now defunct, using external usb 3.0 drives).

Plastic standoffs...will probably have to order those from overseas, Norway doesn't have such a great selection of hardware for custom builds. Can anyone recommend a good UK/NL/DE online shop?

Thanks again...super excited about my es gamble...either I saved 2.5k or shredded 600


----------



## squidman

More chicken and egg...can I set up the raid 0 of the two ssd's in bios (the SATA 6gb ports on the top, c602) and THEN install windows? Sorry had another post, but haven
t figured out how to delete them yet.

good for a laugh....https://www.deal.no/deal/default.asp?page=vare&ProdusentID=A6S90AT

I guess HP gets to do this




























or they posted the wrong pic...


----------



## mking

Hi, has anyone got the Intel Raid working on the SATA ports? If so which Bios?
I tried with 5004 and all I get is LSI setup - Intel doesn't appear with SATA drives - even though I changed motherboard jumper.

Not sure if its a Bios issue or MB RMA time.

I got Intel RAID SCU Option Rom config to display with drives plugged into SCU ports, but can't get SATA ports to display.

I am pretty experienced with RAID and X58 but first time on X79/600 board.

Any help appreciated.

Txs,
Mike


----------



## mking

Ok turns out there's a error in the manual - here is the update !
http://dlcdnet.asus.com/pub/ASUS/mb/LGA2011/Z9PE-D8-WS/Manual/IE7358_Z9PE-D8_WS_Insert_for_web.pdf

The jumper to select Intel rste is not the one in the manual ! Once you use the one in the tech update - all is good.
Boy 8hrs wasted on this one. Mike


----------



## RickSFO

Z9PE-D8 WS. The system was fine until I upgraded the BIOS.

First I upgraded from 0503(.ROM file) to 0703(.ROM). No problem except it reset the SATA from RAID to IDE and it took a while to figure that out. But at least the motherboard started up, and I could get into the bios.

Then I upgraded to 5004(.CAP). Yes, now I know that I should have installed 3109 before going to 5004. Someone just gave me the bios files on a USB drive, and I didn't see the warning on the website until later. Damn.

After upgrading to 5004, it would not boot. Nothing on the screen at all. Code on LED's on motherboard was 25. Then I reset the CMOS memory, and tried again. This time error code 14.

I removed the video card, no difference. Removed all RAM except first slot. Disconnected the CD drive, and all hard drives except one (which is a known good, non-RAID, bootable drive). No difference.

If it matters, I had the bios files on a USB drive and used the EZFlash utility.

So, is the problem because I didn't upgrade in the right sequence? How do I fix it?

UPDATE: Someone else here mentioned that .CAP bios may not work with B0 CPU's (engineering samples). Is that true?

Thanks in advance.


----------



## TheBlademaster01

Quote:


> Originally Posted by *RickSFO*
> 
> Z9PE-D8 WS. The system was fine until I upgraded the BIOS.
> 
> First I upgraded from 0503(.ROM file) to 0703(.ROM). No problem except it reset the SATA from RAID to IDE and it took a while to figure that out. But at least the motherboard started up, and I could get into the bios.
> 
> Then I upgraded to 5004(.CAP). Yes, now I know that I should have installed 3109 before going to 5004. Someone just gave me the bios files on a USB drive, and I didn't see the warning on the website until later. Damn.
> 
> After upgrading to 5004, it would not boot. Nothing on the screen at all. Code on LED's on motherboard was 25. Then I reset the CMOS memory, and tried again. This time error code 14.
> 
> I removed the video card, no difference. Removed all RAM except first slot. Disconnected the CD drive, and all hard drives except one (which is a known good, non-RAID, bootable drive). No difference.
> 
> If it matters, I had the bios files on a USB drive and used the EZFlash utility.
> 
> So, is the problem because I didn't upgrade in the right sequence? How do I fix it?
> 
> *UPDATE: Someone else here mentioned that .CAP bios may not work with B0 CPU's (engineering samples). Is that true?*
> 
> Thanks in advance.


That's true


----------



## RickSFO

Blademaster, can you elaborate on that? Are we talking just a minor incompatibility issue, or that it completely bricks the computer? It's completely dead. I can' t get into the BIOS, nothing. I've never seen anything like it.

I've even tried the Asus "CrashFree BIOS" where you put the bios on a usb drive and plug that directly into the motherboard. Nothing. The drive also had a bootable version of Windows on it.

I tried setting the "ME force recovery mode" jumper to the "recovery" setting. According to the manual "This jumper allows you to force Intel Management Engine (ME) boot from recovery mode when ME become corrupted". I tried both settings - normal and recovery - no difference.

Or maybe the problem is that I didn't upgrade to the 3109 bios before upgrading to 5004, like the website says? But again, it's surprising that it would cause such serious problems.

At this point, I'm completely out of ideas. I've ordered a new bios chip (with the old bios version - 0703).

Thanks for any advice you can provide,

Rick


----------



## TheBlademaster01

Well, I'm still on 0503 but from what PsyQ told me (another member in thos thread). B0 chips won't post on 5004









It is the first BIOS to support Ivy-EP though. It could also be that you didn't flash to 3109 first. I don't really know


----------



## psyq3212

ASUS switched to CAP format with 3xxx BIOS-es.

All 3xxx versions should work with B0 SNB-EP CPUs. You need CAP BIOS if you want to run Windows 8 / Server 2012 since there was a compatibility issue which was only fixed with CAP BIOS-es.

So, for people using B0 SNB-EP and want to run Windows Server 2012, CAP BIOSes v3xxx are OK. Do not go to 5004 as it does not support B0 ES2 samples of SNB-EP anymore.

However, somewhere in the process ASUS also started configuring SPI FLASH settings to "Intel recommended values" - meaning: it is not possible to flash to old version of the BIOS with FTK. People who need to downgrade should keep this in mind - the only ways to downgrade from 5004 are to either buy pre-flashed BIOS chip, or have a SPI-flash programmer and do it yourself (another option is hotflash but this will only work on another ASUS board that does not have the SPI-flash locked)


----------



## RickSFO

Thanks for the useful response psyq3212.

I have ordered a BIOS chip with 0703. Unfortunately the computer is a brick until I get it.

I can't believe ASUS did this. I mean it's one thing if 5004 doesn't work with my CPU, but why isn't there ANY way for me to downgrade the BIOS?!?!?!? I mean the computer won't even POST. And I have read the latest version of the manual, and all the tech support docs on the website. There is nothing that warns of this issue. I've spent 12 hours on this so far.

Thanks again for the information,

Rick


----------



## TheBlademaster01

Now you know, psyq + Andy > Asus


----------



## squidman

Regarding bios...

My 2 x QA8X's (16 USD motherboard repair parts no commercial value) arrived, and I was pretty sure these are B0 stepping parts, but looking at the pcb below in the middle of the serial I see a C0? Anyway to confirm apart from booting up? Still waiting on 2nd H80i fan for cooling for the MONSTER 900d case which I'm really happy I paid the extra dough for! Kicks A$$!

Don't suppose apart from booting there's anyway to verify the bios installed? Didn't see anything on the cmos chip, but then my eyes aren't that good...

Other...should you clear CMOS go through the procedure before first bootup? Switching the jumpers etc? And any opinions on the corsair link cooling thing? Ordered to prevent too many cables going to usb headers and general mess, but not cheap...around 100 bucks.

Again...super stoked! That system in that case is just so badassed....now just need 4x titans es (china where are you?)...


----------



## TheBlademaster01

QAxx is B0/B1 I think. QBxx are the C steppings.

If the board is new, what is the revision?

I just plugged in the chips, filled my loop and ran the system. Bitfenix extension caught fire, plugging in directly to the PSU no problems


----------



## squidman

How do I find out revision number (before plugging in)?

Oh and need plastic standoffs....seen so far suggestions for 1/4 inch tubing and bic pens as diy solutions...can't find them here. Other suggestions? Lastly, it doesn't seem possible or will be a ***** to remove the motherboard tray on the 900d...anyone else with same case? May just install h80i's first, separately install components onto mb, then install mobo onto tray. I assume like other 2011 mobos backplates aren't necessary.

Thanks! Great community here (and very necessary given chronic lack of documentation....case came with 2! pages)......


----------



## TheBlademaster01

You can read my build log in my sig but it was a PITA to install in my case as well... The board is huge and not all screws lined up so even after fighting to get the board in I had to remove it again









About the revision. It's in the lit up area here right from the DIMM slot:


Spoiler: Warning: Spoiler!


----------



## BlackenedTush

Quote:


> Originally Posted by *TheBlademaster01*
> 
> Now you know, psyq + Andy > Asus


I love it! Pretty true.


----------



## squidman

Thanks...I'm going to ghetto-mod plastic standoffs from some masonry screw inserts that happen to 'sort of' fit what 'I think' are mobo screws (no thanks to corsair for documentation!)...I guess as long as the diameter is smaller or doesn't go outside the grounding bits of solder....

One more...do you need to power the ez_molex connecter next to cpu1 for it to boot? Thought that was for extra gpu's, but dearth of info....and where on say a corsair 1200ax would I power it? Assume all the pci-e power outlets are 12v, and you would just get an 8-pin to molex converter? Would suck to be wrong and fry the board or my lovely-jubbly qa8x's! Will try and upload pic of proposed wiring/powering of bits and bobs...

The case rocks, but wow Corsair really sucks ball$ at documentation! Even having purchased 900d, 2xH80i's ax1200 ps, ....etc....they may look sexy, but think I may stay away from them in future....thought this was a US company?


----------



## TheBlademaster01

I don't have the ez_plug wired.


----------



## squidman

One more pic of proposed connections...bit of a big pic, you'll have to zoom around I guess. Think I've sort of! got my head around it....shame there aren't more usb headers! That why I decided on the corsair cooling link, hope it works....

Have something like 7-9 jbod hdd's I'd like to put in there, but may have to rely on some of the externals (don't really need backups in there which are already full....)..right now a right mess of 4! usb 3 hubs running to 9 exernal hdd's and 2x2hdd raid 0 nas's which are long since retired....all running off the one usb 3 port on my little laptop.....







and a 4-port gigabit switch...(migrated to usb 3 a while ago, but back in the day gigabit was the fastest way to have external hdd's!)...Wow this is so overdue....

No how to backup 15TB to the cloud? Preferably for free?


----------



## phillydee

OK been a while since I've hung around these parts but a couple of questions:

I'm currently running RealTemp to monitor temps... anyone with a BETTER suggestion for monitoring temperature of both CPUs?
Also re. RealTemp: a core temp always seem to run 5c hotter. Is this normal? (I'm not too worried, just curious)

Anyone here run After Effects? Any suggestions for good performance settings under "Memory/Multiple CPU" settings?


----------



## BlackenedTush

Squidman,
Quote:


> Originally Posted by *squidman*
> 
> Thanks...I'm going to ghetto-mod plastic standoffs from some masonry screw inserts that happen to 'sort of' fit what 'I think' are mobo screws (no thanks to corsair for documentation!)...I guess as long as the diameter is smaller or doesn't go outside the grounding bits of solder....


The Corsair forum has been very helpful for me and a lot of people. It's connected to their RMA system. I've complained there and had them mail me grommets, screws, and drive carriers for free. I'll bet you could explain your standoff needs and they'd help you out. http://forum.corsair.com/forums/showthread.php?t=117016

Quote:


> Originally Posted by *squidman*
> 
> ...do you need to power the ez_molex connecter next to cpu1 for it to boot?


To get it to boot you need only the one 24 pin ATX cable, and two EPS 12v cables. So there's a total of three power connections from PSU to mobo.

Quote:


> Originally Posted by *squidman*
> 
> The case rocks, but wow Corsair really sucks ball$ at documentation! Even having purchased 900d, 2xH80i's ax1200 ps, ....etc....they may look sexy, but think I may stay away from them in future....thought this was a US company?


Yes lack of docs seems to be more and more common these days. On the Corsair 900 forum that I included above, I saw complaints about that and a post which points back here to Overclock dot Net's 900 thread. The post is a collection & summary of the 900 specs that people had gathered and posted; but you might find various parts of the thread to be helpful. Here it is: http://www.overclock.net/t/1297712/m...#post_19023463.

BT


----------



## Marchitect

Hello All,
I'm glad to join this forum. Unlike many people who join a forum to get info before taking on a build, I have already built a monster using the Asus Z9pe-d8 WS board.
I hope to be able to help others as I have read this board and have gone through many of the troubles people on this board have gone through.
I have been through SO much building my system that I am finally very happy with what I have an again hope to share and help others.
Here's a build list:

MB: Z9pe-d8 WS Bios 3506 - running Q-Code 22 !!
Case: Case Labs MH10 - Acoustically modified
Processor: 2X-2687w
Ram: 64Gb Corsair Dominator Platinum, 2133 - running at 1600
CPU Cooling: 2x Corsair H100i
Case Cooling: Aquaero with 12 fans and 3x PowerAdjust
Power: EVGA Nex 1500
Video 1: EVGA Titan
Video 2: EVGA 680 Classified
Storage: 5x Samsung 840 Pro 512 GB, 2TB HD
USB Expansion: In Case, NZXT USB (IU01)
OS: Win 7 Ult.
Lighting: Custom config

My goal is to post images and possibly videos of my build.
This has been up and running successfully now for 3 months.


----------



## anubis1127

Quote:


> Originally Posted by *Marchitect*
> 
> Hello All,
> I'm glad to join this forum. Unlike many people who join a forum to get info before taking on a build, I have already built a monster using the Asus Z9pe-d8 WS board.
> I hope to be able to help others as I have read this board and have gone through many of the troubles people on this board have gone through.
> I have been through SO much building my system that I am finally very happy with what I have an again hope to share and help others.
> Here's a build list:
> 
> MB: Z9pe-d8 WS Bios 3506 - running Q-Code 22 !!
> Case: Case Labs MH10 - Acoustically modified
> Processor: 2X-2687w
> Ram: 64Gb Corsair Dominator Platinum, 2133 - running at 1600
> CPU Cooling: 2x Corsair H100i
> Case Cooling: Aquaero with 12 fans and 3x PowerAdjust
> Power: EVGA Nex 1500
> Video 1: EVGA Titan
> Video 2: EVGA 680 Classified
> Storage: 5x Samsung 840 Pro 512 GB, 2TB HD
> USB Expansion: In Case, NZXT USB (IU01)
> OS: Win 7 Ult.
> Lighting: Custom config
> 
> My goal is to post images and possibly videos of my build.
> This has been up and running successfully now for 3 months.


Sounds like a nice build, looking forward to seeing some pics.


----------



## BlackenedTush

Quote:


> Originally Posted by *phillydee*
> 
> I'm currently running RealTemp to monitor temps... anyone with a BETTER suggestion for monitoring temperature of both CPUs?


Not sure if these are BETTER; but I like _Core Temp_ a lot for it's straightforward interface and _Piriform Speccy_ is a great tool for overall system info... temps and much more.

Quote:


> Originally Posted by *phillydee*
> 
> Also re. RealTemp: a core temp always seem to run 5c hotter. Is this normal? (I'm not too worried, just curious)


I don't understand what you mean.

Quote:


> Originally Posted by *phillydee*
> 
> Anyone here run After Effects? Any suggestions for good performance settings under "Memory/Multiple CPU" settings?


You're talking about the "Memory and Multiprocessing" preferences and more specifically, within that set of preferences, the "Render Multiple Frames Simultaneously" (RMFS) area.

That's a complicated subject because compositing tools like AE are expected to handle so many types of media and all have their unique peculiarities about which processing suits them. And preview and rendering have their specific peculiarities as well. This performance preferences you're talking about tries to provide flexibility to address all of that. I built my beast by looking at my workflow and reading the Adobe's whitepapers on CS5/CS6 Performance & Autotrol's system requirements for 3DsMax.

*The Theory*
I knew cores, RAM, and fast drives were the tickets to overall flexibility.
As for cores the best I could afford was Xeon E5-2630s. That gave me a total of 12 physical cores (24 _virtual_ or _logical_ cores). I thought about how that applied to those Adobe Preferences you're talking about. Using cores to "render multiple frames simultaneously". I figured that Win 7 could easily run on 2 virtual cores, leaving the other 22 available for AE rendering. Adobe recommended 1GB per virtual core (they call them "CPUs"). So that's 22GB total. AE also allows someone to assign up to 3GB per core (as long as they don't starve the system). I wanted to be able to reach the 3GB/core if I ever found that it would lead to lower rendering times. That meant 66GB of RAM. I also figured that Adobe hyper threading would improve down the line. Plus, I really needed to support my workflow which often included several Adobe and 3D apps being open at the same time. So 64GB was a very good number to shoot for. And I had heard that a good strategy would be to limit the cores assigned to AE to the number of physical cores available. That made no sense to me.

*The reality*

RMFS is always on in my house! It's much faster unless you have a wimpy system.
I've found that the more cores assigned to AE the faster projects are rendered... to a point. Now I've gone well above the 12 core count recommended for my configuration. I've done my max of 22; but improvement does taper off. In fact, I've gotten almost as good performance with19 cores as I do with 22. But I don't do comparison tests with every project. The effectiveness of additional cores can easily vary depending on what media types are in the composite and what type of encoding one is doing. My results are certainly not indicative of all possibilities. For instance, I don't usually do H.264 encoding, which may scale in more linear fashion and use those cores better than Quicktime Photo Jpeg....
I have also found that for most rendering tasks I do, the 1GB per core renders just as fast as 3GB/core, and .75GB/core renders almost as fast as 1GB/core. Generally rendering projects with larger numbers of layers are handled by larger numbers of cores. 3GB per core is only necessary where the media contained in one frame is exceptionally deep and wide (i.e., extreme mega
pixels, 32bit/pixel, and lots of Photoshop layers on one AE layer). At the moment I haven't needed to work that way.
Make sure your _renderer_ is _Classic 3D_ for most projects (Composition Settings > Advanced tab). This is something to consider only if you have successfully registered your NVIDIA GPU in "Preview > GPU Information" preferences, and you have set _Ray-tracing:_ to _GPU_. If so, then it's likely your renderer has automatically been set to _"Ray Traced 3D"_, which is a slower renderer for non 3D work.
*My strategy to find a good basic rendering profile using the max number of cores:*

Set "RAM Reserved for other applications" to 3 GB (just to begin with)
Set "CPUs Reserved for other applications" to 2
Set "RAM allocated per background CPU" to 1.5 GB
Note how many "Actual CPUs used" have been calculated by AE
Go back to "RAM Reserved for other applications" and raise the number of GBs. Stop when the number of "Actual CPUs used" begins to drop.
With the resulting profile you should have more than enough cores and RAM for basic office tasks (running a browser/internet and a word processor) while rendering at most likely the fastest speed possible.

*My strategy to find a basic Adobe CS5/CS6 multitasking Profile:*
Assumes some combination of Bridge, Photoshop, Illustrator, Audition (or Sound Booth) open at same time... and maybe some 3D pre-viewing/rotating in the mix.

Set "RAM Reserved for other applications" to 3 GB (again, just to begin with)
Set "CPUs Reserved for other applications" to 2/3 to 1/2 available cores (this will give other applications, including Adobe apps, some needed cores)
Set "RAM allocated per background CPU" to .75 GB
Note how many "Actual CPUs used" have been calculated by AE
Go back to "RAM Reserved for other applications" and raise the number of GBs. Stop when the number of "Actual CPUs used" begins to drop. This will make RAM available to non-adobe stuff (3D programs, office apps, and system)
Write down the settings and also note how they work. This is a compromise, so you may have to adjust RAM and cores or limit open apps for optimum performance. And of course any rendering task with a 3D application or Premiere, or a heavy Photoshop project would likely warrant shutting down AE and just giving all the system resources to them.

And of course there are other performance strategies within AE we haven't talked about (like SSD caching, preview settings, drive allotment, and ray tracing with GPU, etc)

Your mileage may vary. Hope this helps....

BT


----------



## BlackenedTush

Quote:


> Originally Posted by *Marchitect*
> 
> ...My goal is to post images and possibly videos of my build.
> This has been up and running successfully now for 3 months.


Welcome!
And congrats. Looks like a very nice spec.... lookin 4ward 2 pics as well.
BT


----------



## BlackenedTush

Quote:


> Originally Posted by *squidman*
> 
> ....shame there aren't more usb headers!


If you look at my system in the profile, you'll see that I had to route my superspeed multi-card reader's cables outside of the case & use the USB 3.0 ports... haha


----------



## squidman

Yeah I read you there! Have nowhere to attach front case usb 2 ports (x4) , which is a pain as I'll rarely be going to the back of the case....basically have run out of usb 2 headers...







Still waiting on last fan and corsair link cooling kit (which may or may not work according to forums!)...

Not possible to get a hold of UNC taps and drill bits here, will have to ghetto mod unless I can find some in the UK (assuming standoffs ARE 6-32 UNC)....

For a company based in Cali, Corsair are just as unfriendly info-wise as the Taiwanese....pathetic..thanks for forum links..


----------



## anubis1127

I use i7z to monitor core temps: http://code.google.com/p/i7z/


----------



## phillydee

BT/Anubis: thanks for the system monitoring suggestion. I'll try them and will pick whichever one floats my boat

BT: thanks for your strategy writeup on RMFS--I basically did almost exactly what you did in determining how many cores(CPUs, by AE) are used for rendering frames.

I hit a snag though, that's why I posted my initial question. I had my first reboot due to possible memory leak happen when I had AE use 8 CPUs with 3GB of RAM each. This was the first time my system had apparently crashed, and it seemed at the time for no apparent reason. Previously I had .75mb per CPU and I had (IIRC) 16 CPUs and it seemed fine. Until... I noticed that AE would temporarily freeze/pause before creating the RAM preview. I figured it was RAM starved so I decided to twiddle around a bit with the RMFS settings and when I felt like I had a decent performance boost with 8CPU/3GB RAM the PC suddenly rebooted after 5 minutes of use on this setting... (also my mobo had to be fully drained of power before it would re-POST. I was kinda freaking out when I was troubleshooting this heh)

I'm not on CS6/CC, just on CS5.5. My system has been running so stable with my 3D apps rendering with all 32 threads etc that I was a bit surprised by the sudden reboot. I've turned off RMFS temporarily to try and troubleshoot the issue and decided to post my original question here. I guess I can test my memory out with Memtest or something, but there has been no previous indication that my RAM had any sort of errors. I will probably address the amount of RAM left for other applications as well.

I am running all my CS apps off of my Samsung 840Pro 512GB SSD as well as all of Adobe's temp folders are pointed to the SSD.... I figured this would be a good thing.

I will experiment with less RAM per CPU in AE's RMFS settings, see if that helps.


----------



## squidman

Build update - located only 6-32 UNC tap in Norway! Long trip to boondocks to get it, worked perfectly with suggested drill (metric, I forget...tiny bit smaller than tap).. now just cross fingers that tiny metal bits not floating around and short mobo. Just waiting on last fan. Mobo revision is 1.04, anyone know whether that corresponds to a particular bios? May have to order some flashed eproms (think they're available in NL for around 8 euros each).. haven't been this excited since xmas as a little boy...


----------



## Marchitect

Squid,
If I understand your post, you are looking to expand your usb ports without modifying your case.
Have you seen the Iu01 from NZXT?
I am using 2 of them with great success.

http://www.newegg.com/Product/Product.aspx?Item=N82E16811997024&nm_mc=KNC-GoogleAdwords&cm_mmc=KNC-GoogleAdwords-_-pla-_-Case+Accessories-_-N82E16811997024&gclid=CNeBhZX_-rgCFSU6QgoduTgAng


----------



## squidman

Thanks! Very cool, that could do the trick! But what/how do you mount it/to? I could always use a pcie 3 slot but that seems a waste....need to populate those at some point for cuda/gpgu rendering....









Never mind see you have to velcro it somewhere...although velcro on the back of a pcb, is that kosher?


----------



## squidman

Quote:


> Originally Posted by *phillydee*
> 
> BT/Anubis: thanks for the system monitoring suggestion. I'll try them and will pick whichever one floats my boat
> 
> BT: thanks for your strategy writeup on RMFS--I basically did almost exactly what you did in determining how many cores(CPUs, by AE) are used for rendering frames.
> 
> I hit a snag though, that's why I posted my initial question. I had my first reboot due to possible memory leak happen when I had AE use 8 CPUs with 3GB of RAM each. This was the first time my system had apparently crashed, and it seemed at the time for no apparent reason. Previously I had .75mb per CPU and I had (IIRC) 16 CPUs and it seemed fine. Until... etc......


1) RMFS - what is that? Something I need to know as an AE CS6 user?
2) How many threads/optimization for NukeX? Still learning, in very early stages...but need to go beyond AE at some point (nuke and fusion have proper 3d compositing etc)...aarghh unfortunately the learning curve is VERY steep...


----------



## Marchitect

As of August 9th, Asus has released yet another BIOS - 5103.
Has anyone updated their BIOS to this latest version?

I'm running Bios 3506 and I'm wondering if I should update to 5103. I have extra BIOS chips with older versions on them to use a back up if need be.

My biggest concern is that, after flashing the new bios chip, the same boot options I show now, for some reason may not come up the same on the new bios and require me to make a guess as to what drive to boot from. It's probably a strange concern and nothing to worry about but it came to my mind since my boot sequence is very important and not typical.


----------



## Marchitect

Quote:


> Originally Posted by *squidman*
> 
> Thanks! Very cool, that could do the trick! But what/how do you mount it/to? I could always use a pcie 3 slot but that seems a waste....need to populate those at some point for cuda/gpgu rendering....
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> Never mind see you have to velcro it somewhere...although velcro on the back of a pcb, is that kosher?


Since the USB headers are at the bottom of the MB and that is probably towards the bottom of your case, can't you simply velcro it in that vicinity? You can tuck those USB expansion boards on almost any flat surface.


----------



## squidman

Quote:


> Originally Posted by *Marchitect*
> 
> Since the USB headers are at the bottom of the MB and that is probably towards the bottom of your case, can't you simply velcro it in that vicinity? You can tuck those USB expansion boards on almost any flat surface.


Yep that would do it...but are you using (I assume some sort of glue/sticky stuff) to attach the velcro to the back of the pcb? As it looks like a 'naked' pcb from the newegg pics....i.e. I assume the velcro (one tab) has a 3m/sticky solution for attaching to surfaces? Didn't see any kind of enclosure...never mind it looks like a 2-sided tape is attached to back of pcb, is that right?


----------



## Marchitect

Quote:


> Originally Posted by *squidman*
> 
> Yep that would do it...but are you using (I assume some sort of glue/sticky stuff) to attach the velcro to the back of the pcb? As it looks like a 'naked' pcb from the newegg pics....i.e. I assume the velcro (one tab) has a 3m/sticky solution for attaching to surfaces? Didn't see any kind of enclosure...never mind it looks like a 2-sided tape is attached to back of pcb, is that right?


Here's an example of what I did: Computer is all built now. This is a look at my system in it's very early stages.


----------



## BlackenedTush

Quote:


> Originally Posted by *squidman*
> 
> 1) RMFS - what is that? Something I need to know as an AE CS6 user?
> 2) How many threads/optimization for NukeX? Still learning, in very early stages...but need to go beyond AE at some point (nuke and fusion have proper 3d compositing etc)...aarghh unfortunately the learning curve is VERY steep...


Squidman,

_RMFS_ is just a quicker way to say "Render Multiple Frames Simultaneously"*

BTW, I don't know if we're pushing the boundaries of this thread by talking about how well this MOBO can be utilized for practical work, so anybody... please yell at me if this is distracting...

Nuke Optimization...*
I'm pretty much a noob at Nuke as well, and trying to get my mind around the node-based interface with cryptic attribute panels. At this point I have done a little research on multi-threading/core utilization; but haven't found much to speak of. If they were working that angle, I'd think they'd advertise so we wouldn't have to dig so hard. I have seen a paper on their HPC (High Performance Computing initiative) which makes most efficient use of GPUs and CPU cores, but it seems to be targeted at their plugin products...(and maybe a node here or there in Nuke?).

The Foundry website does have some helpful recommendations about disk caching and optional GPU acceleration on their *System Requirements page*... _It's a start;_ but not what we might have hoped... it just recommends the minimums and leaves it up to your imagination and your personal testing to determine what the maximums might do. For instance: at least a Pentium 4, at least 1 GB of RAM, and a GPU with at least 512MB of VRAM. Not much help there for us _hot rodders_...

I have verified that both disk caching and GPU utilization can be configured to some extent in Nuke(X)'s Preferences... You'll find them in the "Preferences" and "Viewer" tabs. It has a setting, "_cache memory usage_", which may mean RAM - that's puzzling; but promising. I have done absolutely no performance testing. I do plan to be playing with it in the near future.

The _system recommendations page_ seems to talk the most about GPU processing. Here's what I think I know...

Nuke Disk Caching...
It's pretty standard knowledge that disk caching enables image/video applications to process large files with deep-bit color, usually to supplement insufficient RAM. But it's typically at the expense of speed. If one didn't have enough RAM for a project the system would page part of it to the start-up disk, which is slow. A dedicated cache for that is faster and gives one optimization options. But to maintain almost RAM-like performance while paging happens, one would need to employ RAID 0 or 10, SSDs, or a combination. Without testing, it remains to be seen what applying that strategy would accomplish for performance here in Nuke(X).

Nuke GPU Processing...


Nuke can optionally use common GPU technologies to accelerate the viewer. OpenGL 2.0 with support for floating point textures & GLSL (OpenGL shading), plus 512MB+ VRAM. That's old stuff...
GPU processing can accelerate certain nodes (v7); but they don't specify what they are.
For this you'd need NVIDIA technologies. You'd need compute capability of 1.1 or greater, CUDA 4.0 or greater, and the latest drivers. Any Kepler or Fermi card will surpass the requirements. BTW, The *Geek 3D GPU Caps Viewer* utility can show you what technologies your GPU actually has
*Bottom line...*
There's hope for help from the GPU for both viewer acceleration and acceleration of certain nodes.
We can't assume that Nuke will use the sum total of CUDA cores from multiple GPUs (like AE)... or like AE, whether VRAM available to used by each card is limited by the amount of RAM available in the card with the lower amount.
We can probably assume that the more VRAM on our GPU, the higher the texture sizes we can specify in Nuke prefs.
We can always ignore cores and assume a CPU with a faster clock speed will beat an equal one that's slower.
We can probably assume Nuke will use whatever amount of RAM we give it. "_cache memory usage_" in prefs may refer to percentage of system RAM. It can be configured. We don't know for sure what it is or what effect it has at this point.
We can assume that disk caching will help by maintaining performance with big projects; but not necessarily speed things up.
We also know that Nuke isn't arrogant like AE. It can't take command over RAM and cores. So we can't play with their numbers. We can't assume how effective Nuke is at utilizing extra cores or RAM...

*Testing...*
I have a NUKEX match move project coming up in a few weeks; but I only have the PTE version at home. So I'll have to do it on an 8core/6GB MAC Pro with poor caching and GPU.
I do plan to take it home and at some point thereafter test with it. I'll try to get insight on both core and RAM usage with the Windows Resource Monitor. I'll also test caching effectiveness between different cache sizes with RAID10 vs SSD. And I'll play with GPU VRAM settings and texture sizes.

If I dig up more I'll pass it on.

BT


----------



## squidman

Wow thanks for exhaustive reply!

I'm still at the stage where I only just got into bios and got my fans working (turns out it's version 0503, and confirmed my qa8x's actually work...but took multiple dissamblies, switching the graphics card for an ancient radeon 5xxx, reseating ram n times (still sees only 24 of 32gb) etc. Manana I will try and load win7 ultimate from flash stick. Had to remove several dodgy standoffs as well.

What a friggin' drama, but it does sort of look like the black monolith from 2001 space oddyssey standing in my room


----------



## squidman

Ok noob needs yet more help..

Successfully get into win 7 using bootable flash drive to install OS, but windows refuses to install #¤%%!! LSI raid drivers (2 ssd's raided and initiliazed)....no matter which file I point it to. Used downloaded driver files from asus website. Anyone had this problem....

almost there if I can just get these drivers loaded....









Edit: dug out an old floppy drive, following instructions, but win7 ultimate won't accept the drivers because they're not signed....#¤%%&!!









gave up on lsi raid controller, switched over the the intel srt controller....STILL can't install drivers....having raided successfully using intel raid utility in bios


----------



## Marchitect

Quote:


> Originally Posted by *squidman*
> 
> Ok noob needs yet more help..
> 
> Successfully get into win 7 using bootable flash drive to install OS, but windows refuses to install #¤%%!! LSI raid drivers (2 ssd's raided and initiliazed)....no matter which file I point it to. Used downloaded driver files from asus website. Anyone had this problem....
> 
> almost there if I can just get these drivers loaded....
> 
> 
> 
> 
> 
> 
> 
> 
> 
> Edit: dug out an old floppy drive, following instructions, but win7 ultimate won't accept the drivers because they're not signed....#¤%%&!!


I originally had problems trying to install Win 7 Ult with from a flash drive. I was trying to install a stripped down version of Windows, thinking it would be faster with all the unnecessary stuff removed. I used a special program to do this (forgot what it was called) and many people did this with success. However, for me I just couldn't get Windows to load and function properly.
So, in the end, I just purchased a Windows 7 Ultimate disk, installed it from my DVD player and all worked perfectly after that.
It may not be an Asus issue, it could very well be your method of installing Windows.


----------



## squidman

don't actually have a dvd drive on hand (only an old ide drive a old laptop drives) will have to go out and buy one....sigh...

Don't know if that will solve problem, may have to slipstream drivers into win7...what a friggin drama..


----------



## ZDngrfld

Quote:


> Originally Posted by *squidman*
> 
> don't actually have a dvd drive on hand (only an old ide drive a old laptop drives) will have to go out and buy one....sigh...
> 
> Don't know if that will solve problem, may have to slipstream drivers into win7...what a friggin drama..


I had this issue not too long ago and it wasn't an issue with the drivers. It's an issue how the installer handles EFI. I ended up using a dvd to get Server 2008 R2 installed and it worked fine.


----------



## squidman

Was hoping a bios upgrade would sort problem, flashed to 3109 (have b0 QA8X's), only to open a can of worms, cpu's thottled to 1200mhz, endless loop of starting restarting unless cmos was reset every time....
Crossed fingers and flashed to .cap bios 3506...3rd time lucky! Didn't brick it! So off to get a dvd burner...I guess they still have a use after all!







Then just have to deal with gk104 and corsair cooling link nightmares...


----------



## Marchitect

Quote:


> Originally Posted by *squidman*
> 
> Was hoping a bios upgrade would sort problem, flashed to 3109 (have b0 QA8X's), only to open a can of worms, cpu's thottled to 1200mhz, endless loop of starting restarting unless cmos was reset every time....
> Crossed fingers and flashed to .cap bios 3506...3rd time lucky! Didn't brick it! So off to get a dvd burner...I guess they still have a use after all!
> 
> 
> 
> 
> 
> 
> 
> Then just have to deal with gk104 and corsair cooling link nightmares...


Don't worry about the cpu 'throttled' to 1200. That is supposed to happen when you are in the bios if you have Intel's speed step enabled. Do a google search for this, I took this issue way up through Asus' tech support until someone finally who knew what they were talking about knew about Intel Speed Step and explained how it works.

You probably want Speed Step activated as is saves power and doesn't hurt performance. With it activated you will show the CPU's running at 1200 while in bios.


----------



## squidman

weeelll...still in driver hell...picked up an aburdly overpriced dvd-burner(80 friggin bucks!) (samsung sh-224db/rsms sata dvd burner), plopped in a known good dvd install disc without sp1, and of course, get the need driver window again for the dvd drive once win7 starts installing. Now no way to tell if damn dvd is booting in uefi mode, but was hoping that wouldn't matter...otherwise no one would be able to install onto uefi mobos. Of course absolutely no drivers for win7 ultimate for that particular dvd drive (have at least 3! older laptop dvd drives, but no way to connect them! and an older ide dvd drive, ditto)...

"#¤¤%/"#¤¤%&"!!!! and "#&¤#¤/(!! and on the new bios of course 1.2 mhz again...but at least it sees all the corseair vengeance ram (improperly seated, reseated and reinstalled 9th edge standoff using hacksaw method...was stressing the **** out of mobo before!)...well at least I have bios, and h80is work connected directly to my only 2 usb2 headers)...may have to give up on raid for now, but what a PITA!

I was expecting mobo RMA, the dodgy china cpu's to be dead, ram problems...but nothing as friggin stupid as this!


























































all for the lack of working ertified whql drivers!


----------



## squidman

yeah and the stress of this build has given me a friggin' blod clot on my right arm! "#¤¤#%/¤#¤%&/!!!








will try some and burn some other iso's here (not above kat'ing some windows server versions at this point! whatever it freaking takes!) but don't think that's going to resolve the unsigned driver issue....chrissakes! Last resort is to forget raid, install to one of the two samsung 840 ssd's (both installed in the top two sata ports using the CTRL-I intel raid utility), make a disc image, and then somehow rebuild raid after using working drivers (although chicken and egg...I still have to install friggin drivers!).

Phew srry cookie out...just hope I don't have an embolism here (yes doc said it was ok, just a minor case of local thrombosis! whatever...). not going out in a hurry to BUY a copy of W7 pro retail for 500 bucks here (had w7 oed before) , as just used that saved dough to purchase a kick-ass SAGE 2-handed steelhead rod - aka the tcx 7126 'death star' for 50% off...still 600 bucks....









I would purchase IFF they didn't insist on regionally price-discriminating....but nearly 200% is a bit too rich for me.....and no way to update from my lappy's win7 home edition (older acer 5750g which didn't even come with a win dvd!)...or is there from the US websites? nevermind, it will be the dodgy way, can't see how even purchasing will solve my problems (mostly booting from uefi probs)...goddamn, I begin to see why people see the attraction of mapple products..totally brainless, simple, everything works, iff it crashes you just mortgage an arm and a leg and send to overpriced mapple-authorized service centers...

Oh yeah....for those of you initiates of the dark side (fly-fishing)...tight lines and sexy loops!


----------



## squidman

this whole thing just looks like I need to go through with slip-streaming...even the supplied so-called asus help guide should tell you that...wow....at least it's ONLY driver issues at this stage (maybe eeprom or bios issues would be worse, or rma's...but maybe not!)....

links to eeprom flashers and suppliers welcome...ditto whatever other links you want to send. Sorry for spamming, but as you know it happens! when you're at you're at the end of your tether..

cookie out! (or squidman...either one is ok...going back to early days of steam online games)..

Edit: to those who say...go buy a retail win7/8/etc. version...it's not a windows problem, the problem is with lack of driver support and non-existence of certified drivers out there! Tried already with good win7 install dvd...... it could be a uefi bios issue....or mobo being extremely picky needing to match hardware with drivers (but why doesn't win install automatically drivers? for e.g. atapi dvd? etc)..

either way it's a pita...didn't think this build would be simple! It looks like slipstreaming or some variant of f6 install here....iff f6 even works in win7...yes tried from repair menu!'

Edit: fuggedaboutit! I'm just going to install latest version of win server whatever that is....goddammit, M$ does'nt make it easy!


----------



## Johnnywas

So i just built a new workstation with;

2 x XEON E5-2687's
Asus Z9PE-D8
Nvidia Quadro K5000
64GB Ram
Highpoint Rocketraid 3620 raid controller

2 x SSDs in RAID 0 (Marvell controller) FOR OS
2 x SSDs in RAID 0 (marvell controller)
8 x SSDs in pairs of 2 RAID 0 (Raid Controller)

My problem is that windows won't install when the raid card is present. If i remove the RAID controller i can install windows to the marvel RAID set and it will run fine, but as soon as i put the RAID controller in it will cause the system to not boot into windows, it will just sit there with a flashing cursor with nothing to boot from. Even if i try to reinstall windows with the RAID controller in i can't, it won't boot from the CD drive to install.

The raid controller seems to work fine as when it is in you can enter into its bios and create arrays, but it just won't allow OS install or boot, even though i'm installing to the array on the marvel chipset

Any ideas?


----------



## Marchitect

Quote:


> Originally Posted by *Johnnywas*
> 
> So i just built a new workstation with;
> 
> 2 x XEON E5-2687's
> Asus Z9PE-D8
> Nvidia Quadro K5000
> 64GB Ram
> Highpoint Rocketraid 3620 raid controller
> 
> 2 x SSDs in RAID 0 (Marvell controller) FOR OS
> 2 x SSDs in RAID 0 (marvell controller)
> 8 x SSDs in pairs of 2 RAID 0 (Raid Controller)
> 
> My problem is that windows won't install when the raid card is present. If i remove the RAID controller i can install windows to the marvel RAID set and it will run fine, but as soon as i put the RAID controller in it will cause the system to not boot into windows, it will just sit there with a flashing cursor with nothing to boot from. Even if i try to reinstall windows with the RAID controller in i can't, it won't boot from the CD drive to install.
> 
> The raid controller seems to work fine as when it is in you can enter into its bios and create arrays, but it just won't allow OS install or boot, even though i'm installing to the array on the marvel chipset
> 
> Any ideas?


Have you set your Jumpers correctly? There is an update to the Owner's Manual, page 2-28 related to the LSI MegaRAID and Intel RSTe.
That is RAID related and could be your problem.
Just a thought, perhaps it will help.


----------



## Johnnywas

Quote:


> Originally Posted by *Marchitect*
> 
> Have you set your Jumpers correctly? There is an update to the Owner's Manual, page 2-28 related to the LSI MegaRAID and Intel RSTe.
> That is RAID related and could be your problem.
> Just a thought, perhaps it will help.


I tried moving the jumper to the Intel RSTe and then installed windows to the 2 x 120GB Raid 0 (Marvel Chipset) array WITHOUT the RAID card installed and it all worked fine.

Then turned off and installed the RAID card and it won't boot. Even if i direct it to boot from the Marvel array (VD 1) it won't do anything, the motherboard code will just display 'AE'

Completely stumped by this


----------



## squidman

update...3 days later with pita....

LSI megaraid won't support samsung 840 ssd's, and intel rste drivers won't install (tried downloaded f6 drivers from intel site etc..yes aware of jumper settings)...so had to give up on raid completely....

of all the things that could go wrong this is simply moronic....(trying to raid0 the two ssd's in sata ports 1 and 2).....





































apparently many asus customers also on x79 boards had similar problems with the c600 drivers....

addendum....win7 finally installed...has anyone actually got the asus smart caching to work (as opposed to intel srt)`? I'm assuming it works off the 4 grey marvell controlled sataIII ports....can't find a link anywhere to the software (supposedly 1 click)...


----------



## Krobar

Has anyone seen poor FPU performance from C1 stepping ES chips? Integer and memory performance is as expected but I'm only getting 140Gflops from a pair of E5-4617s!


----------



## squidman

I wish I had come that far! Still can't install win7! Got as far as SP1, then the next updates resulted in bsod....sigh...back to square one after 4 days (not even getting raid to work!)....after 4 hrs updating...








Maybe open-box mobo and b0 stepping wasn't the way to go...but I really had to do this on a budget and was working up until the last updates! Godda#¤%&!!it...

edit: I made sure to remove my os ssd from the hot-swap cage, as that is another potential problem.....

edit2: my gk104 (gtx 770 4gb) doesn't work at all either...relying on a radeon 4700 or whatever it is...whether in pcie 1 or 3....I have to say I had a bad feeling about this mobo after reading the newegg reviews.....it was too good to be true....but I got greedy...







May have to learn linux at this rate....understand why mapple people feel superior....

sigh have 2 days in which I can return components....undo all watercooling, etc......all the cabling and zip-ties etc......sigh sigh....for a while I had it going! see task manager...

edit last of this post...would be nice to be able to see/edit the html for editing....on your menu bar....thanks admin!


----------



## TheBlademaster01

Yeah, I think you have a faulty board. I never have BSODs in Windows. I'm running overclocked as well.


----------



## Krobar

Squidman,

If you reinstall Windows and don't run updates does it work again?

Here is a little example of my FPU performance problem (Second bar is Dual E5-2687W and third bar is Dual E5-2620)




One other thing. Can the Z9PE-D8 set DDR3-1866? I was surprised to find the Supermicro board will let me OC the RAM and will run DDR3-1866 with Sandybridge. Would not let me increase the QPI beyond spec though.


----------



## squidman

I haven't had any problems forcing my corsair vengeance to 1600....and all the updates went fine until the next batch after sp1......took friggin almost 4 hrs of course. Will pick up a legit copy of win7 tomm. (another major expedition to find an OEM version for sale!) but the iso's I've used so far have had no probs on other pc's (iff this offends your delicate sensibilities, tune to another channel! - after all a lot of us use es chips!)...blechh the regional price-discrimination of m$ doesn't do anything for me (500 bucks here for retail version). No the win-to-flash thing and rebus thing didn't work (not surprising given uefi probs)...on known good iso's. most likely an oem win7 pro won't be better, but will give it a try (still 350 bucks)....

Of course the lsi and intel raid controllers are another matter...had to give up there! At least we know that LSI refuses to support samsung ssd's....so can count them out....On reinstall n! here, forget....

Props to the forum....wow it's great to have SOME KIND of support!









Yeah krobar, on my nth reinstall, have not run updates and it works! So may have to install paragon or something (used acronis for a while...but they are still buggy as hell!) to make a rescue disc.. I got to sp1 on win7 ultimate last time, and then it all went pear shaped. Haven't disabled pxe, or messed too much with things (been lots of cmos clears, and know the bios and raid setups inside and out by now!)....so yeah frustrating. Wondering whether to install the samsung magician ****, but that would probably just make things worse. Will try installing updated asus drivers from usb instead of from mobo dvd this time, and then up to sp1.

Unfortunately iff you're on gk104's and b0 stepping (i.e. can't go above a certain bios) it looks like your screwed. Will have to return my zotac gtx 770 4gb, and sit with my piddly radeon 46xx for now. Oh well, I didn't really have the dough for proper e5-2687w's anyway! 32 threads are pretty cool anyway...live and learn! But it does sort of work the dodgy es thing! Was hoping I could maybe game a little bit, oh well.....


----------



## Johnnywas

Quote:


> Originally Posted by *Johnnywas*
> 
> So i just built a new workstation with;
> 
> 2 x XEON E5-2687's
> Asus Z9PE-D8
> Nvidia Quadro K5000
> 64GB Ram
> Highpoint Rocketraid 3620 raid controller
> 
> 2 x SSDs in RAID 0 (Marvell controller) FOR OS
> 2 x SSDs in RAID 0 (marvell controller)
> 8 x SSDs in pairs of 2 RAID 0 (Raid Controller)
> 
> My problem is that windows won't install when the raid card is present. If i remove the RAID controller i can install windows to the marvel RAID set and it will run fine, but as soon as i put the RAID controller in it will cause the system to not boot into windows, it will just sit there with a flashing cursor with nothing to boot from. Even if i try to reinstall windows with the RAID controller in i can't, it won't boot from the CD drive to install.
> 
> The raid controller seems to work fine as when it is in you can enter into its bios and create arrays, but it just won't allow OS install or boot, even though i'm installing to the array on the marvel chipset
> 
> Any ideas?


I may have solved this. I found a support doc from highpoint that says some motherboards won't boot if the raid controller and on board raid controllers are enabled. If you press END when the raid controller bios first appears during boot then it will skip loading the RAID controller bios and you can then boot into windows.

Obviously pressing END each time isn't effective so i'm updating bios now for the permanent fix. Will post back to let you all know how it goes in case anyone else has the same issue in the future


----------



## squidman

Quote:


> Originally Posted by *TheBlademaster01*
> 
> Yeah, I think you have a faulty board. I never have BSODs in Windows. I'm running overclocked as well.


I can technically still send back to the online webstore...but now that everything is installed, a major pita. so really really reluctant (2h80i's, all the cabling zip-tied etc....) and it seems to work except for these goddamn driver issues and the last bsod! Even the memory seems to work ok (but need to do a memtest off a floppy here)....

thanks for the moral support!


----------



## squidman

uhhmm all I can say is props on the scores! All I could afford was a pair of dodgy e5-2665's from prc...but wow glad I went there (originally going 2x e5-2620's.....) just have to get it to work! Oh yeah will have to get rid of the one 2620 somehow...any takers? Not an ES of course, proper pukkah...







in original box etc....unopened...


----------



## Johnnywas

Quote:


> Originally Posted by *Johnnywas*
> 
> So i just built a new workstation with;
> 
> 2 x XEON E5-2687's
> Asus Z9PE-D8
> Nvidia Quadro K5000
> 64GB Ram
> Highpoint Rocketraid 3620 raid controller
> 
> 2 x SSDs in RAID 0 (Marvell controller) FOR OS
> 2 x SSDs in RAID 0 (marvell controller)
> 8 x SSDs in pairs of 2 RAID 0 (Raid Controller)
> 
> My problem is that windows won't install when the raid card is present. If i remove the RAID controller i can install windows to the marvel RAID set and it will run fine, but as soon as i put the RAID controller in it will cause the system to not boot into windows, it will just sit there with a flashing cursor with nothing to boot from. Even if i try to reinstall windows with the RAID controller in i can't, it won't boot from the CD drive to install.
> 
> The raid controller seems to work fine as when it is in you can enter into its bios and create arrays, but it just won't allow OS install or boot, even though i'm installing to the array on the marvel chipset
> 
> Any ideas?


So, i managed to fix this issue after A LOT of head scratching and cursing.

For anyone who faces the issue where they have a RAID card installed in their system which is causing it to not be able to boot into windows or boot from a windows disk to install windows then this was the fix.

To allow yourself to initially get past the problem, Press END when the RAID card bios screen first appears to skip the RAID card BIOS from loading. This will then allow you to boot into windows.

Then, you will need to create a DOS bootable USB with the firmware and bios update for your RAID card on it. Boot into this and update the BIOS and Firmware for the RAID card.

Then enter into the BIOS for the RAID card. If you intend to boot from the host adapter card, disable the option listed as "Reallocate EBDA".If you do not need to boot from the host adapter card, disable both the "Reallocate EBDA" and "INT13" options.

This solved it for me


----------



## Johnnywas

The only issue i'm having now is that it seems to be really slow to boot after a restart.

If you restart the system from windows it seems like it can take up to 5 minutes or more to boot into windows. Cold start seems to be fine.

Any ideas?


----------



## ZDngrfld

I sure am glad I'm just using this board in one of my dedicated folding rigs with the amount of problems you guys are having getting everything working properly with Windows...


----------



## TheBlademaster01

I really don't have that much of an issue in Windows. Suse has been the greater disappointment for me. Maybe an issue is not getting the RAM over 1600MHz, but I'm pretty happy with the board. Haven't tried a new GPU in here though. Just the old 470, wondering if I should upgrade. Otherwise the new card is going in my 2600K rig.


----------



## psyq3212

JFYI, Z9PE D8 WS BIOS 5103 contains production microcode update for IvyTown CPUs. Yay









Another thing, for 12 core Ivy Bridge EP CPUs, you should disable Isoc in the QPI configuration.

My system is now fully stable, even with NVIDIA Titan running @PCIe 3.0 link speed.


----------



## Krobar

Quote:


> Originally Posted by *psyq3212*
> 
> JFYI, Z9PE D8 WS BIOS 5103 contains production microcode update for IvyTown CPUs. Yay
> 
> 
> 
> 
> 
> 
> 
> 
> 
> Another thing, for 10/12 core Ivy Bridge EP CPUs, you should disable Isoc in the QPI configuration.
> 
> My system is now fully stable, even with NVIDIA Titan running @PCIe 3.0 link speed.


Congratulations. Was your systems stable with PCI-e 3.0 on Titan with SB-EP?

Also any idea what ISOC actually does? Supermicro recommends enabling it for HEDT and Audio platforms; does not seem to effect DPC on my setup.


----------



## TheBlademaster01

Quote:


> Originally Posted by *psyq3212*
> 
> JFYI, Z9PE D8 WS BIOS 5103 contains production microcode update for IvyTown CPUs. Yay
> 
> 
> 
> 
> 
> 
> 
> 
> 
> Another thing, for 10/12 core Ivy Bridge EP CPUs, you should disable Isoc in the QPI configuration.
> 
> My system is now fully stable, even with NVIDIA Titan running @PCIe 3.0 link speed.


Still breaks B0 SNB-EP compatibility right?


----------



## psyq3212

Quote:


> Originally Posted by *Krobar*
> 
> Congratulations. Was your systems stable with PCI-e 3.0 on Titan with SB-EP?
> 
> Also any idea what ISOC actually does? Supermicro recommends enabling it for HEDT and Audio platforms; does not seem to effect DPC on my setup.


Isoc is a mode of transmission between two QPI nodes (Isochronous traffic). It should be ON for 2P configurations, but it should be disabled for 12 core IvyTown 2P configurations. For CPUs with 10 cores and less, it should be ON in 2P configurations.

For HEDT platforms, it only make sense for the old Nehalem/Westmere CPUs which used QPI for connection with the IOH.

SNB-E and IVB-E HEDT parts do not use QPI links at all, as QPI is only used for connection between multiple CPUs and there is only one CPU, so it does not really matter for those parts.

I could not pair TITAN on my old SNB-EP @ PCIe 3.0 as I had B0 and C0 steppings in the past.

--

As for B0 SNB-EP I do not have those anymore so unfortunately I cannot test, but I am almost 100% sure it does not work since there were too many MRC changes for ASUS to bother to restore compatibility









@edit - I made a small mistake, Isoch should be OFF for 12-core parts only.


----------



## TheBlademaster01

Did the Titan even work with the B0 chips (though at 2.0)?


----------



## ignoramus

Quote:


> BIOS 5103 for Z9PE-D8 WS released to public
> 1. Separate BIOS item "Enhanced Intel SpeedStep Technology" and "Turbo Mode".
> 2. Update RSTe to 3.7.0.1049. (Legacy Option ROM and UEFI driver)


Does this mean we can downgrade from cap to rom if we update to bios 5103? (i can build and overclock a system but im not overly technical - that makes me a 'noob'!)

I pretty new to this thread/forum but i have a z9pe-d8 on the way for my budget rendering system built to get me out of trouble during my masters of architecture study for the next 2 years. I have also got 8x8GB of PC3-10600 1333mhz ram (bought for £80 - boom!) a 650 watt cpu from my current fx-8150 build and an large atx case which _should_ be big enough to contain this monster board - im not concerned with cable mgmt but the unused drive bays lend themselves to it.

Im still mmmming and ahhhing over which ESs to pick up. the current options are the B0 stepping 2650 2-2.3ghz or the B0 stepping 2665 2.3-2.5ghz. Both are about the same price, it seems stupid to go for the 2650s but B0s are all i can afford atm. With the blademaster not reaching the level of performance he expects with the 2665 i'm still checking fleabay every day for some miracle listing similar to this http://www.ebay.co.uk/itm/271253189374?ssPageName=STRK:MEWAX:IT&_trksid=p3984.m1438.l2649 because i have a feeling they would have worked and i would be enjoying 20c/40t for an insanely cheap price. (someone tell me im wrong or ill cry!)

edit- the identification discussion of the chips above is here if youre interested. bottom of the first page is what youre looking for. http://www.cpu-world.com/forum/viewtopic.php?t=21333


----------



## psyq3212

Quote:


> Originally Posted by *ignoramus*
> 
> Does this mean we can downgrade from cap to rom if we update to bios 5103? (i can build and overclock a system but im not overly technical - that makes me a 'noob'!)


Unfortunately, no - Option ROM is just a part of the BIOS/EFI and it is related to the RAID controller. What ASUS is saying is, that they upgraded the option ROM.
Quote:


> Originally Posted by *TheBlademaster01*
> 
> Did the Titan even work with the B0 chips (though at 2.0)?


Yes it did - but it could not pair @PCIe 3.0


----------



## RickSFO

OK, got the new bios chip with 0703 and now the system is up again.

However, it is still unstable, same as before. I was hoping the newer bios would fix this. I do long (overnight) optimization runs in a cad program, and the computer always locks up after while. Sometimes it makes it several hours, sometimes only one hour. It's not a BSOD, it just locks up. Sometimes it tries to reboot itself, but if it gets that far then it always hangs during the Windows logo animation. In any case, I always have to power cycle the machine to get it back up. It's odd that it doesn't crash in exactly the same way every time...

System:
Asus Z9PE-D8
Windows 7 64bit Ultimate
Bios 0703 with all default settings
Two Xeon 2687W B0 (engineering samples)
4x4GB Gskill RipjawsZ DDR3-1600-PC3-12800 (F3-12800CLQ-16GBZL)
Samsung 830 256GB SSD (I did install the Samsung drivers from the CD that came with the drive)
ATI Firepro V4900

Two things I'm wondering:

1) Memory settings. The spec for the memory is 9-9-9-24-2N. But if I set the bios to 9-9-9-24, the system will not boot (says overclock failed). So I have to set it back to default which is 11-11-11-24. Note that I don't know what the "2N" means or how to set it in the bios.

2) Are there any bios settings I need to change for the B0 cpu's?

Thanks in advance,

Rick


----------



## TheBlademaster01

I don't think so. My system is stable out of the box (2x B0 2665s) and 104MHz BUS and 1600MHz RAM is stable. Any difference in timings makes the system fail to boot (same case as you). Even when loosening timings







...

2N implies a command rate (interval) of 2 clock cycles between adressing the memory cell and the actual command. 2N/2T is best for stability since it gives the IMC and RAM enough slack to sample data while it's valid. 1N/1T is fastest since it halves the delay (only one clock cycle of slack) but data might not be valid if the RAM or IMC has too much clock skew.


----------



## ZDngrfld

I've had issues with memory on this board from the get go. It's picky about what it likes. I'm currently running 8 1GB 1333MHz Hynix sticks. It didn't like the G.Skill memory I originally wanted to run in it...


----------



## DizZz

Quote:


> Originally Posted by *ZDngrfld*
> 
> I've had issues with memory on this board from the get go. It's picky about what it likes. I'm currently running 8 1GB 1333MHz Hynix sticks. It didn't like the G.Skill memory I originally wanted to run in it...


Hmm I'm running this kit quad channel with no problems.

http://www.newegg.com/Product/Product.aspx?Item=N82E16820231427


----------



## TheBlademaster01

I have Dominator Platinums 1866 C9. It can't run 1866 but again, no problems otherwise. I'm on BIOS 0503 though so that could also be part of the reason.


----------



## psyq3212

Now that I got my system stable with IVB-EP I was trying to play with memory.

I have eight M393B2G70BH0-YK0 ECC DIMMs, nominally rated @1600 MHz and 1.35v (so-called "Green Memory" by Samsung, otherwise known for great overclockability)

With SNB-EP B0 and C0/C2, I was able to run this RAM stable @1866 MHz with 1.5v and @2133 MHz @1.65v (which is above maximum Xeon VDDR spec!)

With Ivy EP (admittedly, the HCC version 2697 v2) I am able to run it stable @1866 MHz and 1.5v, as expected.

Unfortunately, I cannot get the board to post at all @2133 MHz with IVB-EP. It is stuck in the loop 00 - 19 - AF - 00, indicating memory initialization failure









At this moment, I am not sure if this is the limit of the CPU samples I have (could be), or ASUS still did not "fix" the MRC code to allow proper memory overclocking above Intel spec. I will try to get in contact with ASUS and volunteer to test beta code if needed. Ivy Bridge EP uncore is actually quite different compared to Sandy Bridge EP uncore, so it could well be that ASUS did not finish their MRC modifications yet for overclocking.

In other news, setting BCLK to 104 MHz works without problems.


----------



## ZDngrfld

Quote:


> Originally Posted by *psyq3212*
> 
> Now that I got my system stable with IVB-EP I was trying to play with memory.
> 
> I have eight M393B2G70BH0-YK0 ECC DIMMs, nominally rated @1600 MHz and 1.35v (so-called "Green Memory" by Samsung, otherwise known for great overclockability)
> 
> With SNB-EP B0 and C0/C2, I was able to run this RAM stable @1866 MHz with 1.5v and @2133 MHz @1.65v (which is above maximum Xeon VDDR spec!)
> 
> With Ivy EP (admittedly, the HCC version 2697 v2) I am able to run it stable @1866 MHz and 1.5v, as expected.
> 
> Unfortunately, I cannot get the board to post at all @2133 MHz with IVB-EP. It is stuck in the loop 00 - 19 - AF - 00, indicating memory initialization failure
> 
> 
> 
> 
> 
> 
> 
> 
> 
> At this moment, I am not sure if this is the limit of the CPU samples I have (could be), or ASUS still did not "fix" the MRC code to allow proper memory overclocking above Intel spec. I will try to get in contact with ASUS and volunteer to test beta code if needed. Ivy Bridge EP uncore is actually quite different compared to Sandy Bridge EP uncore, so it could well be that ASUS did not finish their MRC modifications yet for overclocking.
> 
> In other news, setting BCLK to 104 MHz works without problems.


I was running four of their non-ecc Green Memory sticks at stock speeds without a hitch. It seems to favor them more than any other sticks I've thrown at it. Any chance you could run [email protected] and get us some numbers with those insane chips?


----------



## TheBlademaster01

I've seen CineBench scores of around 30 with 2697s. That would put it 10-15% lower than 4x 6386. Though the QRB will skew things a bit.


----------



## DizZz

Quote:


> Originally Posted by *ZDngrfld*
> 
> Any chance you could run [email protected] and get us some numbers with those insane chips?


THIS PLEASE


----------



## psyq3212

I'd rather wait few more weeks for the detailed benchmark results, sorry. I'd still like to be able to work with the manufacturer hehe









But I can tell you this much - with heavy threaded workload with all CPUs 95-100% busy I am getting ~1.4x (slightly higher than 1.4x) improvements compared to dual 2687W set-up.

And this is with DDR3 running @1866 MHz on both systems and the same motherboard, so it is really the CPU difference.

1.4x is nothing short of impressive I must say.


----------



## TheBlademaster01

What do you get in Cinebench. Rough number is enough


----------



## psyq3212

Ok, here is the quick result - I am not sure if everything is optimal, as I just ran the test with 48 threads.

I am getting 34.34 pts


----------



## TheBlademaster01

Okay, that's actually faster than 4P 6386 systems









Must have seen ES2 chips then. They turbo'd to 2.8 and had 12/24 QDxx chips (listed E5-2697v2). Sell me those chips


----------



## psyq3212

Yep, those are ES2.

QS/Retail goes all the way up to 3 GHz for all-core turbo (as already pointed out by TH http://www.tomshardware.com/reviews/ivy-bridge-ep-xeon-e5-2697-v2-benchmarks,3585-2.html







)

It became more or less the practice that ES2 chips have somewhat lower frequencies and turbo bins compared to the QS/Retail.

But, like I said on the other thread - if you want all the cores Ivy can do and also high clocks, wait for Ivy Bridge EX. As this is the EX line which represents absolutely the best Intel does, they will probably







have parts clocked considerably higher than EP line.

Unfortunately, the cost of the retail ones will be comparable to a good kidney. But I am sure they will start leaking on Ebay in few months. Think about it - 8S configuration with 15 cores, that's 120 cores and 240 threads of pure goodness. With high clocks as well.


----------



## DizZz

Quote:


> Originally Posted by *psyq3212*
> 
> Ok, here is the quick result - I am not sure if everything is optimal, as I just ran the test with 48 threads.
> 
> I am getting 34.34 pts
> 
> 
> Spoiler: Warning: Spoiler!


Can you run [email protected] pleeease?


----------



## TheBlademaster01

I don't have Ivy-EX money and my applications don't scale well over that. But considering a 3.3GHz 64 core AMD Piledriver system scores 33p in Cinebench and I get somewhat over 19p I'd be glad with even a 2692v2 lol.

@DizZz

Think 800-900k PPD









On P8102/8103 that is.


----------



## Johnnywas

Has anyone had problems with slow booting?

The system is generally fine when it boots from a cold start but if you restart the system from windows or reset button it will take 5-10+ minutes to boot if it does at all.

Specs are:
2 x XEON E5-2687's
Asus Z9PE-D8
Nvidia Quadro K5000
64GB Ram
Highpoint Rocketraid 3620 raid controller

2 x SSDs in RAID 0 (Marvell controller) FOR OS
2 x SSDs in RAID 0 (marvell controller)
8 x SSDs in pairs of 2 RAID 0 (Raid Controller)


----------



## Marchitect

Quote:


> Originally Posted by *psyq3212*
> 
> But, like I said on the other thread - if you want all the cores Ivy can do and also high clocks, *wait for Ivy Bridge EX. As this is the EX line which represents absolutely the best Intel does, they will probably
> 
> 
> 
> 
> 
> 
> 
> have parts clocked considerably higher than EP line.
> *
> Unfortunately, the cost of the retail ones will be comparable to a good kidney. But I am sure they will start leaking on Ebay in few months. Think about it - 8S configuration with 15 cores, that's 120 cores and 240 threads of pure goodness. With high clocks as well.


Hopefully this isn't to stupid of a question, but will these processors you describe above be compatible with the 2011 socket and thus compatible on the Z9PE-D8 motherboard? I'm impressed what has happened with the performance over my 2687w's in less than 1 year and hopefully in the future if and when I feel the need to jump to these incredible performance processors I won't have to change the MB.


----------



## TheBlademaster01

It should be socket 2011, but whether or not they support it is an interesting question indeed


----------



## derickwm

For anyone interested I have a pair of 2687Ws for sale


----------



## psyq3212

Quote:


> Originally Posted by *Marchitect*
> 
> Hopefully this isn't to stupid of a question, but will these processors you describe above be compatible with the 2011 socket and thus compatible on the Z9PE-D8 motherboard? I'm impressed what has happened with the performance over my 2687w's in less than 1 year and hopefully in the future if and when I feel the need to jump to these incredible performance processors I won't have to change the MB.


Unfortunately, no.

EX processors use LGA2011-1 socket, which is different than LGA2011. It has same number of pins, but they are keyed differently.

I suppose this is mainly because different memory support (EX platform will also support DDR4).

So different motherboard will be needed for sure. On the good side, launch EX platform will also be compatible with future Haswell EX CPU.


----------



## TheBlademaster01

Kind of in doubt derick, especially with IVB here


----------



## derickwm

Yeah, main reason for selling them. Just thought I'd put them just in case.


----------



## TheBlademaster01

You also want those 2697s eh







?


----------



## arvidab

Who doesn't?

psyq: What's the power draw on IVB-E look like during Cinebench or similar?


----------



## TheBlademaster01

My guess would be very similar to 2687Ws


----------



## psyq3212

I do not have the SNB-EP here, but the draw is some ~20-30W less than dual-2687W I used to have. Temperatures are also lower by approx 10 degrees C.

However, 2687W is a workstation CPU with 150W nominal TDP - so it is not really a fair comparison SKU-wise.

Although, performance per watt wise, Ivy Bridge EP is simply impressive - you get ~1.4x power of 2687W for slightly less power draw. What is not to like









Performance per dollar is roughly the same I suppose. This is the result of a total lack of competition







It is really a pity AMD is incapable of making a competitive CPU in the high-performance segment.


----------



## derickwm

Quote:


> Originally Posted by *arvidab*
> 
> Who doesn't?


^^^


----------



## TheBlademaster01

Yeah, the chips also released very late. This is just Intel playing boss, you're awesome for giving this information btw psyq









So I'd imagine, 370-380W from wall in CineBench which is pretty similar to Folding maybe even a bit more because of the visual part.


----------



## ignoramus

Quote:


> Originally Posted by *derickwm*
> 
> For anyone interested I have a pair of 2687Ws for sale


Inboxed with my budget. But it's probably too low if yours aren't ES.


----------



## derickwm

Quote:


> Originally Posted by *ignoramus*
> 
> Quote:
> 
> 
> 
> Originally Posted by *derickwm*
> 
> For anyone interested I have a pair of 2687Ws for sale
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> Inboxed with my budget. But it's probably too low if yours aren't ES.
Click to expand...

Hmm? I didn't get a PM from you? Nvm


----------



## Krobar

Hmmm, looks like Inter-core efficiency is weird on my setup too:


----------



## squidman

Victory is mine!!!! Actually got 770 GTX working (pcie slot 4)....









Now JUST have to get smart-caching and "#¤¤#%!! cooling link and nodes to work....knock on wood...


----------



## TheBlademaster01

Smart caching is working without me wanting it I believe. It fills up my SSD needlessly though


----------



## Marchitect

Quote:


> Originally Posted by *psyq3212*
> 
> Unfortunately, no.
> 
> EX processors use LGA2011-1 socket, which is different than LGA2011. It has same number of pins, but they are keyed differently.
> 
> I suppose this is mainly because different memory support (EX platform will also support DDR4).
> 
> So different motherboard will be needed for sure. On the good side, launch EX platform will also be compatible with future Haswell EX CPU.


So, with the Asus Z9PE-D9 ws board and my 2687w processors, am I already at the max potential (processor wise) for this board?
Everything else the Intel comes out with that has a higher performance value will not be compatible with the Z9PE board? I was hoping that the board would take some future processors in the event I want to up grade.

I really liked the idea of keeping my build in tact and just upgrading processors when the time comes. I guess that idea is out the window.
Please confirm my suspicions.


----------



## TheBlademaster01

The best CPU for this board will be 2697v2 or arguably 2687Wv2. IVB-EP will be compatible. IVB-EX will not.

IVB-EP will be the new Xeon E5 and IVB-EX will be the new Xeon E7.


----------



## psyq3212

Quote:


> Originally Posted by *Marchitect*
> 
> So, with the Asus Z9PE-D9 ws board and my 2687w processors, am I already at the max potential (processor wise) for this board?
> Everything else the Intel comes out with that has a higher performance value will not be compatible with the Z9PE board? I was hoping that the board would take some future processors in the event I want to up grade.
> 
> I really liked the idea of keeping my build in tact and just upgrading processors when the time comes. I guess that idea is out the window.
> Please confirm my suspicions.


As TheBlademaster01 already said, you can upgrade to Ivy Bridge EP - the best you can get is Xeon 2697 v2 >or< 2687W v2, depending on what you prefer (core count vs. max clock).

With Xeon 2697 v2 you will get ~40% performance improvement with no increases in power draw on the same socket - not bad for an upgrade.

Unfortunately, Ivy Bridge EP is the end of the line for Z9PE-D8 WS. Haswell EP comes with its own LGA 2011 socket version, so new board will be needed.

I do hope that C610 successor of Z9PE-D8 WS will have 16 DDR4 slots. This is pretty much the only thing I lack with Z9PE-D8 WS.


----------



## Krobar

Quote:


> Originally Posted by *psyq3212*
> 
> As TheBlademaster01 already said, you can upgrade to Ivy Bridge EP - the best you can get is Xeon 2697 v2 >or< 2687W v2, depending on what you prefer (core count vs. max clock).
> 
> With Xeon 2697 v2 you will get ~40% performance improvement with no increases in power draw on the same socket - not bad for an upgrade.
> 
> Unfortunately, Ivy Bridge EP is the end of the line for Z9PE-D8 WS. Haswell EP comes with its own LGA 2011 socket version, so new board will be needed.
> 
> I do hope that C610 successor of Z9PE-D8 WS will have 16 DDR4 slots. This is pretty much the only thing I lack with Z9PE-D8 WS.


Would you accept an HPTX sized replacement?


----------



## derickwm

I would love a new HPTX sized board to replace my SR-2 :drool:


----------



## TheBlademaster01

Don't want


----------



## psyq3212

Quote:


> Originally Posted by *Krobar*
> 
> Would you accept an HPTX sized replacement?


I have no issues with the mobo size so, yes - I would.

I am not even sure if the size increase would be needed - Haswell EP will move some of the voltage regulation circuitry to the CPU, so less components on the motherboard.

In fact, I was also quite happy with Z9PE-D16 except for one thing - no overclocking options. I do not care about CPU overclocking, since it is locked-out anyway for 2S/4S LGA 2011 Xeons, but memory overclocking is important for me - especially since Samsung DIMMs have no issues running @1866 and even @2133 MHz. But, with Z9PE-D16, it is not possible to control the timings in any way.

Funny enough, Z9PE-D16 did allow booting ECC RAM @1866 MHz, but the configured timings were really funky.


----------



## TheBlademaster01

Size increase most likely won't be needed but I already had issues filling 8DIMMs so I won't really need 16 either


----------



## futchi

The existence of 2687W V2 is nearly meaningless comparing to 2667 V2.

E5-2687wv2 [email protected] 150W TDP, 25MB L3, turbo bin: 2 2 2 2 3 4 5 6
E5-2667 v2 [email protected] 130W TDP, 25MB L3, turbo bin: 3 3 3 3 4 5 6 7

So 2667 v2 has same frequency as 2687wv2 with turbo boost, while 20W lower and about 100$ cheaper


----------



## TheBlademaster01

2687W v2 is 10/20


----------



## derickwm

Nope 2687W V2 is 8/16. Confirmed from the inside 

Initial sources had it wrong.


----------



## futchi

From intel internal documents


----------



## futchi

Some SKUs of e5 2600 v2 have more than 2.5MB L3/core.

2687W v2 & 2667 v2: 8c 25MB L3
2643 V2: 6c 25MB L3
2637 V2: 4c 15MB L3


----------



## TheBlademaster01

So 2687W is slower than 2680 this time around? Which genius came up with that, oh boy this is going to be fun


----------



## psyq3212

Yep, 2687W v2 is 8 core unfortunately - the document that futchi posted is a DCL that accompanies CPU batches sent to OEMs.

By the way, I do not think it is a very good idea to post snapshots of those documents as they are confidential. I doubt Intel will send black helicopters for leaking the CPU spec this late, but even that is a breach of a NDA and Intel might start harassing overclock.net and ask for takedowns which nobody wants.

By the way, Ivy EP CPUs start appearing on Ebay:

Auction #121149112884


----------



## futchi

2687W V2 is certainly lower than 2680v2, it may be even slower than 2670v2.

2680 v2 [email protected], all core turbo: 3.1GHz 3.1*10=31
2670 V2 [email protected], all core turbo:2.9GHz 2.9*10=29
2687Wv2 [email protected], all core turbo:3.6Ghz 3.6*8=28.8


----------



## Krobar

Quote:


> Originally Posted by *futchi*
> 
> 2687W V2 is certainly lower than 2680v2, it may be even slower than 2670v2.
> 
> 2680 v2 [email protected], all core turbo: 3.1GHz 3.1*10=31
> 2670 V2 10[email protected], all core turbo:2.9GHz 2.9*10=29
> 2687Wv2 [email protected], all core turbo:3.6Ghz 3.6*8=28.8


I would consider the higher 150W TDP to be a good thing for sustained turbo providing you have decent cooling.


----------



## derickwm

These processors run pretty cool from what I've experienced. 2687Ws at max all core turbo maxed at like mid 50s during Folding with a single 480 rad.


----------



## Krobar

Quote:


> Originally Posted by *psyq3212*
> 
> I have no issues with the mobo size so, yes - I would.
> 
> I am not even sure if the size increase would be needed - Haswell EP will move some of the voltage regulation circuitry to the CPU, so less components on the motherboard.
> 
> In fact, I was also quite happy with Z9PE-D16 except for one thing - no overclocking options. I do not care about CPU overclocking, since it is locked-out anyway for 2S/4S LGA 2011 Xeons, but memory overclocking is important for me - especially since Samsung DIMMs have no issues running @1866 and even @2133 MHz. But, with Z9PE-D16, it is not possible to control the timings in any way.
> 
> Funny enough, Z9PE-D16 did allow booting ECC RAM @1866 MHz, but the configured timings were really funky.


Strangely enough my X9-DRI allow RAM OC too, I really did not expect it. My 1333 Reg sticks are fine at 1600 but only just get in to Windows at 1866; timings looked like Jedec defaults (I think it ignores SPD if RAM timings are forced).

Ever tried the X9DAX (Not suitable if you are a SLI user):
http://www.supermicro.com/products/motherboard/Xeon/C600/X9DAX-iTF.cfm


----------



## ignoramus

ok so im at a loss. how does one search ebay by auction # alone??

edit- nevermind. worked it out!

edit again- isnt that just an auction for a E5-2670?


----------



## squidman

Ok back again with another V for VICTORY by the total newb....







..

corsair link cooling working and both h80i's visible as well as all case fans, commander, cooling node etc (with caveat..on startup you need to let the h80's die on their a$$, then unplug and replug sata power to them...!!!




























) but ok THEN it works.

So next challenge.....the 4 marvell ports (grey SATA 6GB), I would really like to put my 2x 60gb el-cheapo Kingston ssd's on those with 2 WD Green 3TB drives for smart-caching....prbly another nightmare. Even better iff I can get the ASUS smart-caching (as opposed to intel's rste) to work there....they advertise it on the mobo box, and website, but no links to the software! Supposedly one-click! What gives? Has anyone managed that? It's supposed to be some proprietary form of marvell's smart-caching (same but with asus front-end)...

No embolism here yet! And WOW getting the gtx 770/aka the gtx 680 to work was a huge relief.....although on pcie 4 I have a Tesla jumper or something, no idea what that's for...in position 2, should I switch it back? (maybe not, it's not broke!)...









Edit those 12 v molex from the PS...are they DC? How bad would it be (like a car battery?) if I got shocked when undoing and redoing the damn corsair link power connections? (not like I want to find out!)

Edit 2 and 3: it actually turned on without doing the repowering thing to the h80i's....wow! But on the other hand, a cpu score of 17.78 on cinebench...wow that sucked baahss...!!!! Still better than my lappy, but that score is barely above a 3930k! What the??????


----------



## DizZz

So for [email protected], 2697 v2 is going to be the best option?


----------



## TheBlademaster01

Yeah









By far.


----------



## Krobar

Quote:


> Originally Posted by *squidman*
> 
> Ok back again with another V for VICTORY by the total newb....
> 
> 
> 
> 
> 
> 
> 
> ..
> 
> corsair link cooling working and both h80i's visible as well as all case fans, commander, cooling node etc (with caveat..on startup you need to let the h80's die on their a$$, then unplug and replug sata power to them...!!!
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> ) but ok THEN it works.
> 
> So next challenge.....the 4 marvell ports (grey SATA 6GB), I would really like to put my 2x 60gb el-cheapo Kingston ssd's on those with 2 WD Green 3TB drives for smart-caching....prbly another nightmare. Even better iff I can get the ASUS smart-caching (as opposed to intel's rste) to work there....they advertise it on the mobo box, and website, but no links to the software! Supposedly one-click! What gives? Has anyone managed that? It's supposed to be some proprietary form of marvell's smart-caching (same but with asus front-end)...
> 
> No embolism here yet! And WOW getting the gtx 770/aka the gtx 680 to work was a huge relief.....although on pcie 4 I have a Tesla jumper or something, no idea what that's for...in position 2, should I switch it back? (maybe not, it's not broke!)...
> 
> 
> 
> 
> 
> 
> 
> 
> 
> Edit those 12 v molex from the PS...are they DC? How bad would it be (like a car battery?) if I got shocked when undoing and redoing the damn corsair link power connections? (not like I want to find out!)
> 
> Edit 2 and 3: it actually turned on without doing the repowering thing to the h80i's....wow! But on the other hand, a cpu score of 17.78 on cinebench...wow that sucked baahss...!!!! Still better than my lappy, but that score is barely above a 3930k! What the??????


I though a 3930K was only good for about 10 Cinebench points at stock.


----------



## TheBlademaster01

Quote:


> Originally Posted by *squidman*
> 
> Ok back again with another V for VICTORY by the total newb....
> 
> 
> 
> 
> 
> 
> 
> ..
> 
> corsair link cooling working and both h80i's visible as well as all case fans, commander, cooling node etc (with caveat..on startup you need to let the h80's die on their a$$, then unplug and replug sata power to them...!!!
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> ) but ok THEN it works.
> 
> So next challenge.....the 4 marvell ports (grey SATA 6GB), I would really like to put my 2x 60gb el-cheapo Kingston ssd's on those with 2 WD Green 3TB drives for smart-caching....prbly another nightmare. Even better iff I can get the ASUS smart-caching (as opposed to intel's rste) to work there....they advertise it on the mobo box, and website, but no links to the software! Supposedly one-click! What gives? Has anyone managed that? It's supposed to be some proprietary form of marvell's smart-caching (same but with asus front-end)...
> 
> No embolism here yet! And WOW getting the gtx 770/aka the gtx 680 to work was a huge relief.....although on pcie 4 I have a Tesla jumper or something, no idea what that's for...in position 2, should I switch it back? (maybe not, it's not broke!)...
> 
> 
> 
> 
> 
> 
> 
> 
> 
> Edit those 12 v molex from the PS...are they DC? How bad would it be (like a car battery?) if I got shocked when undoing and redoing the damn corsair link power connections? (not like I want to find out!)
> 
> Edit 2 and 3: it actually turned on without doing the repowering thing to the h80i's....wow! But on the other hand, a cpu score of 17.78 on cinebench...wow that sucked baahss...!!!! Still better than my lappy, but that score is barely above a 3930k! What the??????


Quote:


> Originally Posted by *Krobar*
> 
> I though a 3930K was only good for about 10 Cinebench points at stock.


I get 17.94 or so out of the box. With a little bit of overclocking (2.5GHz) and RAM at 1600 I get 19.24. A stock 3930K gets 10.5p, my 5.2GHz 2600K also get 10.5 and a 5GHz 3930K/3960X gets 14.5p with a lot higher power draw. I think C0 2665s get 20.5 or 21p or so.


----------



## psyq3212

Quote:


> Originally Posted by *ignoramus*
> 
> ok so im at a loss. how does one search ebay by auction # alone??
> 
> edit- nevermind. worked it out!
> 
> edit again- isnt that just an auction for a E5-2670?


Nope, QF7P is E5-2630 v2 (http://www.cpu-world.com/CPUs/Xeon/Intel-Xeon%20E5-2630%20v2.html)

Quote:


> Originally Posted by *Krobar*
> 
> Strangely enough my X9-DRI allow RAM OC too, I really did not expect it. My 1333 Reg sticks are fine at 1600 but only just get in to Windows at 1866; timings looked like Jedec defaults (I think it ignores SPD if RAM timings are forced).
> 
> Ever tried the X9DAX (Not suitable if you are a SLI user):
> http://www.supermicro.com/products/motherboard/Xeon/C600/X9DAX-iTF.cfm


Unfortunately, X9DAX does not work for me due to minimal number of PCIe x16 slots.

Right now, with Supermicro you have to either choose decent number of PCIe x16 slots or overclocking options, which is a bit stupid IMO. I hope Supermicro will expand their range of boards with "Hyper-Speed", which is their own name for overclocking. However, even with "Hyper-Speed", the overclocking options are still not as elaborate as with ASUS, but I suppose the board is much more stable and free of quirks compared to ASUS Z9PE-D8 WS.


----------



## ignoramus

Quote:


> Originally Posted by *TheBlademaster01*
> 
> I get 17.94 or so out of the box. With a little bit of overclocking (2.5GHz) and RAM at 1600 I get 19.24. A stock 3930K gets 10.5p, my 5.2GHz 2600K also get 10.5 and a 5GHz 3930K/3960X gets 14.5p with a lot higher power draw. I think C0 2665s get 20.5 or 21p or so.


do you think that score would go up if you filled all of your ram slots? is that still the question?

i just ordered the same B0 stepping 2665s as you and i wont be complaining with a ~20 score considering im jumping from a ~7.5 with my fx-8150 build. i have 64gig (8*8) at the ready for when everything does finally show up!

have you done this:
Quote:


> Also, if you're folding with them setting the C-state to no limit, Energy performance to performance mode and long duration power limit to 130w they should maintain 2.8ghz turbo clocks all the time.


from this thread. it helps keep the turbo speed active.


----------



## TheBlademaster01

Yes my score went up from 18 to 19.24 when I switched to quad channel and clocked the RAM to 1600MHz.

B0 only turbo up to 2.4GHz all core though. 2.5 if you increase bClk to 104MHz. 2.6GHz for single thread performance.


----------



## beaker7

Kind of disappointed if 2687w v2 is indeed 8 core. What is the point of that SKU in that lineup? Makes no sense.


----------



## psyq3212

It is, indeed, a bit puzzling - what it offers is not really considerably better than E5-2667 v2 (both will go to 3.6 GHz for all 8 cores).

I presume E5-2667 v2 will stay in the highest turbo state less than 2687W v2, as it is a 130W part.


----------



## squidman

My bad regarding 3930k...was looking at 3960x oc to 4.5+ ghz. Still no replies on smartcaching? H80is back to needing repower while system is on....living dangerously.


----------



## ViperJPB

Quick note and question for the board.

Love this motherboard and build. Running 2x E5-2687Ws (16 cores, 32 cores hyperthreaded), 32GB of ram, 2x Corsair GT3 240GB SSDs, a 1TB WD Black storage drive, 3x GTX 680 4GB cards, 1 R680 SAS Raid with an 8TB internal RAID, and one 1 Blackmagic Decklink Extreme 3D+ IO card.

Scores: 25.56 in Cinebench, 42,545 in Geekbench 2, 31.99 in Sandra 2012, and in SLI with one PhysX 16,674 3DMark12.

Have had zero issues with the board or the initial build and setup. Primary use is for work, lots of 3D rendering, After Effects, and Color Grading using CUDA. Does all of the above like a champ. The only gotcha is 32GB is totally not enough to feed these memory hungry processors. After Effects wants at least 2GB a core in multithreaded rendering or it will drop to a single core. Next upgrade is definitely a minimum of 128GB of ram.

Been running strong for a year and then just this week one of the two Intel(R) 82574L Gigabit Network controllers stopped working. Device Manager reports a Code 10 and says the device cannot be started. Tried uninstalling and updating multiple drivers to no avail. This occurred right after a power outage due to a lightning storm. Was in the middle of a render on all cores, and the video cards were hovering at 80C. I had yet to purchase the $1K plus APC required to power this beast and have been living on the edge running it soley off a Monster Surge Protector. I am thinking it was either the abrupt shutdown that caused the built up heat to fizzle that controller (from some googling it appears Intel Gigabit controllers are super susceptible to Code 10 and failures, Realtek not so much.). Or simply a surge that somehow got through to the system and luckily only effected that port. Odd thing is I'm not even using that port and have nothing plugged into it. Only item that isn't on a surge protector was a powered usb hub...however it and all devices work. Computer performs the same...just no dice on activating that port in device manager. Due to my work schedule, and reading the nightmares on this forum, I don't dare try to update the bios until my schedule and deadlines are clear. Plus a re-install of windows or re-instating of a backup is risky with deadlines looming. Anybody else have issues with a gigabit controller? Was it the storm or just a a weak controller...or both? Any thoughts would be appreciated.


----------



## beaker7

Quote:


> Originally Posted by *ViperJPB*
> 
> Quick note and question for the board.
> 
> Love this motherboard and build.=.


I do similar work with a similar build and have never had a problem, which makes me almost nervous. 2x E5-2687w's with 128 GB 1600mhz ECC.

Planning another similar build and trying to talk myself out of going Supermicro instead. Unfortunately no one has the same PCI-E layout as this one.


----------



## ViperJPB

Quote:


> Originally Posted by *beaker7*
> 
> I do similar work with a similar build and have never had a problem, which makes me almost nervous. 2x E5-2687w's with 128 GB 1600mhz ECC.
> 
> Planning another similar build and trying to talk myself out of going Supermicro instead. Unfortunately no one has the same PCI-E layout as this one.


With 128GB of ram you shouldn't have my After Effects problem of dropping out of Multi-Processor rendering. I would be interested to know if you did as I plan to move to 128GB soon which should rectify the memory starved cores if I'm right.

I really think it was the crash or the lightning storm that fried out one of my Intel gigabit controllers. Plus from what I read the Intel ones are really sensitive. I wouldn't worry too much as long as you have it on a good UPS or APC.

Supermicro has some nice layouts and is what ProMax is using on their ProMax+ workstation I almost bought. Thing is I like to game in my spare time and keep a separate windows install on this system just for that. No SLI on Supermicro server boards







.

Building another!? This rig was pricey, I wish I could build two!


----------



## ViperJPB

Quote:


> Originally Posted by *beaker7*
> 
> I do similar work with a similar build and have never had a problem, which makes me almost nervous. 2x E5-2687w's with 128 GB 1600mhz ECC.
> 
> Planning another similar build and trying to talk myself out of going Supermicro instead. Unfortunately no one has the same PCI-E layout as this one.


Granted I only have the AE Multi-Processor issue when working with a comp that has multiple uncompressed 4K layers.


----------



## ignoramus

Quote:


> Originally Posted by *TheBlademaster01*
> 
> I get 17.94 or so out of the box. With a little bit of overclocking (2.5GHz) and RAM at 1600 I get 19.24. A stock 3930K gets 10.5p, my 5.2GHz 2600K also get 10.5 and a 5GHz 3930K/3960X gets 14.5p with a lot higher power draw. I think C0 2665s get 20.5 or 21p or so.


Do you think your score might increase a small amount if you updated the BIOS to the latest it will take?

Also do you have any information on power draw for your setup?

Thanks


----------



## TheBlademaster01

I don't really think that it will increase with later BIOS versions.

And yeah. At stock and full load on only the CPUs it draws ~300W from the wall. At 2.5 it draws somewhat below 320W. Idle is 150-170W from wall.


----------



## nhcoohrh

I have two Z9pe-d8 WS motherboards and to be honest I don't they are great thing but find 2 CPU mobs that could have a dedicated GPU it's not easy in the market so I had to choose for this.

I concluded that ATI GPUS with this motherboard it's a nightmare, NVIDIA work a little better but still new models PCI 3.0 only could work in SLOT 3.

The best solution was to use Quadro 4000, it's old PCI connection so it works in any PCI ex slot.

But now I have a problem with my second Motherboard and need an opinion.

I tried to update to latest bios with ez flash, result was that now motherboard doesn't boot anymore. It stucks with code QS 14 (pre memory CPU).

- Clear CMOS didn't worked.
- Asus Intel ME Firmware recovery didn't worked. (this jumper in the draw of the manual it's the same jumper for the RAID select, so I don't know even if it exists or not).

Anyone have a clue how to recover the BIOS of this, or I just do RMA of the motherboard and get other.


----------



## ignoramus

Are you using Engineering Sample chips? Older stepping chips aren't supported by the newer BIOS revisions.

Downgrading your BIOS isn't possible but at the very least you can order a new bios chip that comes with the factory BIOS version and swap it out. Another member in this thread did the same thing to 'unbrick' his board.


----------



## BlackenedTush

Quote:


> Originally Posted by *ViperJPB*
> 
> With 128GB of ram you shouldn't have my After Effects problem of dropping out of Multi-Processor rendering. I would be interested to know if you did as I plan to move to 128GB soon which should rectify the memory starved cores if I'm right.
> 
> I really think it was the crash or the lightning storm that fried out one of my Intel gigabit controllers. Plus from what I read the Intel ones are really sensitive. I wouldn't worry too much as long as you have it on a good UPS or APC.


Nice Rig Viper! I really envy the peepz like you and _beaker7_ who can afford the E5-2687Ws. I never trusted the more affordable ES chips and with the complexity of the board, I guess it's a good thing... . My 6-core 2.3GHz E5-2630s are my weakest link, but I do have 64GB RAM.

Sorry about your zapped controller. I have the _CyberPower 1500PFCLCD UPS_. Since I've owned it I am able to continue tackling school deadlines right on through thunderstorms with reckless impunity. And it's saved my butt a handful of times this past year. It's a fraction of the $1k you are talking... I'm really happy with it.

With my config I can max out AE's _Render Multiple Frames Simultaneouly_ by applying 22 of my 24 virtual cores @ 1GB/core, or I can use 19 cores @ 3GB/core. So far with the projects I've done the difference between them in rendering efficiency is fairly insignificant. So I'm not seeing any symptoms of starving. Granted, my workflow doesn't yet involve 4k with 10-bit+ colorspace. Actually for 8-bit stuff, .75GB/core has been almost as fast. So I have way more RAM than AE seems to need. Most of what my 64GB of RAM buys me is a flexible multi-tasking CS6 workflow.

Now it's interesting reading your comments about starving your processors and even experiencing a situation where AE drops out of multi-processor rendering. I've never heard of these issues. Given my personal experience I would have thought that you could apply 28 cores @ 1GB/core and scream through most rendering tasks.
What are the symptoms of dropping out of MP rendering? And how does one know when one's cores are starved for RAM?

BT


----------



## ViperJPB

Quote:


> Originally Posted by *BlackenedTush*
> 
> Nice Rig Viper! I really envy the peepz like you and _beaker7_ who can afford the E5-2687Ws. I never trusted the more affordable ES chips and with the complexity of the board, I guess it's a good thing... . My 6-core 2.3GHz E5-2630s are my weakest link, but I do have 64GB RAM.
> 
> Sorry about your zapped controller. I have the _CyberPower 1500PFCLCD UPS_. Since I've owned it I am able to continue tackling school deadlines right on through thunderstorms with reckless impunity. And it's saved my butt a handful of times this past year. It's a fraction of the $1k you are talking... I'm really happy with it.
> 
> With my config I can max out AE's _Render Multiple Frames Simultaneouly_ by applying 22 of my 24 virtual cores @ 1GB/core, or I can use 19 cores @ 3GB/core. So far with the projects I've done the difference between them in rendering efficiency is fairly insignificant. So I'm not seeing any symptoms of starving. Granted, my workflow doesn't yet involve 4k with 10-bit+ colorspace. Actually for 8-bit stuff, .75GB/core has been almost as fast. So I have way more RAM than AE seems to need. Most of what my 64GB of RAM buys me is a flexible multi-tasking CS6 workflow.
> 
> Now it's interesting reading your comments about starving your processors and even experiencing a situation where AE drops out of multi-processor rendering. I've never heard of these issues. Given my personal experience I would have thought that you could apply 28 cores @ 1GB/core and scream through most rendering tasks.
> What are the symptoms of dropping out of MP rendering? And how does one know when one's cores are starved for RAM?
> 
> BT


Thx! Yeah it's a great system so far. After Effects officially recommends 2GBs of ram per core minimum for anything HD. Everything above they recommend 3GBs per. I'm usually working at 16bit color depth with 1080P and 4K tiff files off a RED Epic camera. Most comps with many duplicate layers of similar Rez files. With 3GBs of ram per core AE reports using only 8 cores with my 32GBs of ram. When it drops to a single core you can see it in the info panel. With Multi-Processor rendering it will say MP....when it drops to one it usually says "switching to foreground renderer" and after switching the MP will be missing, meaning its only spinning on one core.


----------



## psyq3212

Ivy Bridge E (HEDT) die shots: http://wccftech.com/intel-ivy-bridge-e-power-density/

This is the desktop version, and the die shot clearly shows that it is based on a native 6-core design, unlike SNB-EP. So, I guess some of the HEDT CPU buyers would not think they got a crippled 8-core.

Now, what I would >really< like to see is a shot of a HCC Ivy Bridge EP die (12 core models).

Why? Well, I am really wondering if HCC is actually a cut-down 15 core design or if it is a native 12-core one.

Inquiring minds want to know


----------



## BlackenedTush

Quote:


> Originally Posted by *ViperJPB*
> 
> Thx! Yeah it's a great system so far. After Effects officially recommends 2GBs of ram per core minimum for anything HD. Everything above they recommend 3GBs per. I'm usually working at 16bit color depth with 1080P and 4K tiff files off a RED Epic camera. Most comps with many duplicate layers of similar Rez files. With 3GBs of ram per core AE reports using only 8 cores with my 32GBs of ram. When it drops to a single core you can see it in the info panel. With Multi-Processor rendering it will say MP....when it drops to one it usually says "switching to foreground renderer" and after switching the MP will be missing, meaning its only spinning on one core.


"*Switching to guns Mav!*..."

Thanks for that. I hadn't monitored the info panel during rendering before. I have noticed that at the bottom of the AE window during rendering that the "RAM used" never has gone above 12 percent for my stuff. It makes perfect sense that with deep & wide Tiff file sequences that you are busting AE's per frame requirements.

I have some strongly mixed thoughts about Adobe's Creative Cloud paradigm... but that's another story...
Anyway, I see that AE CC has gone above CS6's maximum of 3GB/core. They've added 4GB and 6GB settings. Supposing you got your 128GB and AE CC, you could theoretically use 30 @ 4GB/core or drop it down to 18 cores @ 6GB/core, without AE "switching to guns". And you wouldn't have enough RAM to employ all your cores! What a kick!

I just read about their support of network rendering farms with a CS6 or CC license (AErender.exe). Like folding, that is another fascinating application of multi-core processing.

BT


----------



## ViperJPB

Quote:


> Originally Posted by *BlackenedTush*
> 
> " *I see that AE CC has gone above CS6's maximum of 3GB/core. They've added 4GB and 6GB settings.
> 
> BT*


Switching to guns...nice. That's good to hear about the CC AE settings allowing for more ram per core. I wish they would allow you to set a max per core, as this is a minimum. Would be nice if you knew you didn't have enough ram for all cores to assign 24GBs of ram between 4 or 2 cores...etc.


----------



## BlackenedTush

Quote:


> Originally Posted by *ViperJPB*
> 
> Switching to guns...nice. That's good to hear about the CC AE settings allowing for more ram per core. I wish they would allow you to set a max per core, as this is a minimum. Would be nice if you knew you didn't have enough ram for all cores to assign 24GBs of ram between 4 or 2 cores...etc.


I agree sir.
And don't getr me started. There are other intelligent ways they could offer the AE user a more efficient application of their own hardware...

It's the nature of compositing to combine a myriad of media types. AE, Nuke, et al programatically know the nature of any imported source material...It's necessary in order to be able to programmatically apply the specific processing methodologies that each needs for both preview and rendering.

Now Adobe doesn't go into much detail as to the specific processing requirements underneath these media types or likewise, behind the plugin effects that can be applied; but they do throw out their disclaimers in their RMFS discussions. One can build a general picture from a wide range of these and other online discussions; but even for the most studious note takers among us, it's still glorified gray magic setting up preferences for a render job or to be intelligent about which FX plugins to use in a composite. Some use GPU technologies like OpenCL, OpenGL, Cuda... some CPU, Some like more RAM... some like more processors some hyperthread well, some not... and some aren't limited to just one type of processing. So for example, I saw a user on Creative Cow who was depending on MP to rip thru a project which contained the "_Element 3D_" effect. But once rendering started, MP got kicked off and things crawled (because that particular plugin actually needed the GPU). We as users aren't privvy to such programmatic processing decisions - we just suffer the _After Effects_ (pun intended). rolleyes:

That said, there's no practical reason why Adobe (or a 3rd party programmer using the API) couldn't build a _"performance analysis module"_ which takes that info and uses to build a list of all composites and FX included within a project, and then matches each item with its particular processing requirements. What a tactical advantage it could give users. In its most simple form, each item could be preceded by a check box that the user could check/un-check. From the checked list the tool would generate the optimum/recommended preference settings for the combination of all checked items. The user could optionally click a button to directly "apply" these recommended settings to their prefs. Such a tool could help a user determine which content might benefit from separate processing or pre-processing... or help them determine FX choices that may be better handled with alternates... It could help them determine how to best approach an impending deadline and gain more predictability. A display with a log function would also provide valuable strategic advantage by pointing out which hardware components have historically been the weak links in the configuration.
@ lloyd mcclendon... you're a programmer... U interested? haha

Just sayin...

BT


----------



## beaker7

I currently have the Asus with E5-2687w's but I'm reaaaly leaning towards the Supermicro X9DA7 for the next build on Ivy Bridge-EP. Haven't had any problems with current Asus but I wonder if i just happened to get a cherry one and don't want any problems. This is a work machine. Am I being paranoid?

Plus with Ivy also not being unlocked the O/C features are kind of pointless. Also like the horizontal ram configuration since I set my workstations on their side.


----------



## ViperJPB

Yup Yup. Synthetic aperture for instance is my go to color effect in AE. Regardless of your comps color depth settings, that plugin processes at 32bit. Multiply that effect throughout your comp and you are adding quite a bit of render time and processing to your comp. Element 3D is a fantastic plugin but extremely hamstrung by it's GPU only processing. It does not support MP by design. Caused some big slow downs in render times for my current project. Big comp with lots of subcomps, HD, and 4K layers...add Element and now you have to process all of that on one core. Best to probably pre-render the comp when possible sans the elements that have to interact with Element. AE....old code....with lots of tacked on expansions and revisions.


----------



## Krobar

Quote:


> Originally Posted by *beaker7*
> 
> I currently have the Asus with E5-2687w's but I'm reaaaly leaning towards the Supermicro X9DA7 for the next build on Ivy Bridge-EP. Haven't had any problems with current Asus but I wonder if i just happened to get a cherry one and don't want any problems. This is a work machine. Am I being paranoid?
> 
> Plus with Ivy also not being unlocked the O/C features are kind of pointless. Also like the horizontal ram configuration since I set my workstations on their side.


Hi Beaker,

If you want overclocking and intend to use a matching Supermicro chassis you should possibly take a look at this:
http://www.supermicro.nl/products/system/4U/7047/SYS-7047AX-TRF.cfm

Asus trend to be a touch better performing and better at supporting S3 Acpi in my experience but Supermicro have decent phone support (At least in Europe) and as I mentioned earlier a wider selection of boards.


----------



## beaker7

Quote:


> Originally Posted by *Krobar*
> 
> Hi Beaker,
> 
> If you want overclocking and intend to use a matching Supermicro chassis you should possibly take a look at this:
> http://www.supermicro.nl/products/system/4U/7047/SYS-7047AX-TRF.cfm
> 
> Asus trend to be a touch better performing and better at supporting S3 Acpi in my experience but Supermicro have decent phone support (At least in Europe) and as I mentioned earlier a wider selection of boards.


Thanks. I've already gotten a Lian-Li A75 for this build. Deciding between another Asus Z9PE-D8, the X9DA7, or perhaps even the X9DRG-QF if it will fit in that case. SM's Hyperspeed boards are ok but the PCI-E layout is way insufficient. With locked multipliers I'm not that concerned about O/C as BCLK gains are minimal and for me PCI-E stability is key.


----------



## psyq3212

Latest ThrottleStop 6.00 beta supports IvyTown CPUs.

I use TS to enforce max-frequency for certain workloads where SpeedStep would negatively impact the performance.

And, yes, those temps are on air


----------



## squidman

Somewhat off-topic: hard drive reliability...

Looking to populate the 4 SAS/Sata ports with hdd's, possibly in raid 1+0/10 (after catastrophic POS 3TB seagate external went RAW on me). Any recommendations for hard drives? Can't afford proper enterprise sata or sas drives. Swore I'd stay off seagate this time, but they have really cheap 4tb consumer drives at the moment. Otherwise looking at the WD Red or WD NAS 3tb drives (all of which get mixed reviews on newegg, with many DOA). Anyone with views on reliable make? Whether mtbf or other metrics? And has anyone had bad/good experiences with the intel RSTe/SCU controller?

cheers


----------



## ViperJPB

Quote:


> Originally Posted by *psyq3212*
> 
> Latest ThrottleStop 6.00 beta supports IvyTown CPUs.
> 
> I use TS to enforce max-frequency for certain workloads where SpeedStep would negatively impact the performance.
> 
> And, yes, those temps are on air


Very Interesting App!.....Hmm....wondering what I can juice out of my 2687Ws. Though under a long Prime95 run I am just below the hottest I would want my procs to go.


----------



## Migonacci

Wanted to rehash an issue I've been having with my CS6 suite being reset to trial (license gets lost) after I wake my computer from S3 sleep state. This occurs without having cs6 apps open and putting the computer to sleep. I've tried s1 sleep and I dont experience this problem, so its something to do with how the comp suspends to ram. Has anyone else had issues with S3 sleep state?

Specs:

dual Xeon E5 2670

64gb Gskill PC3-10700


----------



## squidman

Quote:


> Originally Posted by *ViperJPB*
> 
> Yup Yup. Synthetic aperture for instance is my go to color effect in AE. Regardless of your comps color depth settings, that plugin processes at 32bit. Multiply that effect throughout your comp and you are adding quite a bit of render time and processing to your comp. Element 3D is a fantastic plugin but extremely hamstrung by it's GPU only processing. It does not support MP by design. Caused some big slow downs in render times for my current project. Big comp with lots of subcomps, HD, and 4K layers...add Element and now you have to process all of that on one core. Best to probably pre-render the comp when possible sans the elements that have to interact with Element. AE....old code....with lots of tacked on expansions and revisions.


I haven't used element3d yet, but for larger projects, is there any time saving over the traditional route of comping in layers (matchmove-export to 3d program-export various layers-comp back into ae or nuke or fusion...)`? If element slows the render to a crawl? I must confess I often just 'eyeball it' once in say max, and use the tracking points as a general reference when combining live action and cg...and eyeball it again when back in after effects. But then I'm more of a hobbyist (no 4k camera here yet!)....


----------



## psyq3212

Quote:


> Originally Posted by *ViperJPB*
> 
> Very Interesting App!.....Hmm....wondering what I can juice out of my 2687Ws. Though under a long Prime95 run I am just below the hottest I would want my procs to go.


Normally, if your application is stressing all cores with more than, say, 90% of C0 residency, the EIST should keep all cores at the highest P state possible (which is, in most cases, the highest turbo state for workstation systems).

However, if the workload is fluctuating, sometimes EIST is too aggressive with clocking-down, and you could get better results by forcing the cores to the highest P state.

This is not recommended to be done all of the time, because the power consumption will go up by 10-20W. But for short periods, it can help in specific workloads.

Normally, it is also possible to tune Windows promotion/demotion thresholds for P-state transitions, but I prefer ThrottleStop.

Btw, for people who want to learn more about Windows CPU power management configuration options see this:

http://download.microsoft.com/download/3/0/2/3027D574-C433-412A-A8B6-5E0A75D5B237/ProcPowerMgmtWin7.docx

Like I said, I prefer ThrottleStop - but for server admins, Windows own configuration has advantages since it is part of the system policy.


----------



## psyq3212

@edit - double post, sorry


----------



## ignoramus

My friend has a pair of 2648L CPUs and is now wondering if they will work in the Z9PE-D8 WS even though it is not on the cpu support list. However, on various ebay auctions it is listed as 2648L/2650, which makes us think it might work. It is an 'embedded' cpu so we aren't too hopeful, anyone know?

Thanks


----------



## BlackenedTush

Quote:


> Originally Posted by *psyq3212*
> 
> Normally, if your application is stressing all cores with more than, say, 90% of C0 residency, the EIST should keep all cores at the highest P state possible (which is, in most cases, the highest turbo state for workstation systems).
> 
> However, if the workload is fluctuating, sometimes EIST is too aggressive with clocking-down, and you could get better results by forcing the cores to the highest P state.
> 
> This is not recommended to be done all of the time, because the power consumption will go up by 10-20W. But for short periods, it can help in specific workloads.
> 
> Normally, it is also possible to tune Windows promotion/demotion thresholds for P-state transitions, but I prefer ThrottleStop.
> 
> Btw, for people who want to learn more about Windows CPU power management configuration options see this:
> 
> http://download.microsoft.com/download/3/0/2/3027D574-C433-412A-A8B6-5E0A75D5B237/ProcPowerMgmtWin7.docx
> 
> Like I said, I prefer ThrottleStop - but for server admins, Windows own configuration has advantages since it is part of the system policy.


psyq3212

I appreciate the insight and the resource. I've been looking for understanding on this for quite some time, and somewhat unsuccessfully.

BT


----------



## beaker7

Getting antsy for the Ivy EPs to come out....got everything else ready to go except memory and CPUs


----------



## anubis1127

Has anybody had any luck running newer PCI-E 3.0 GPUs on this board? Does the newer BIOS revisions fix the issues I read about early on? I'm just using the on-board VGA header for display right now, but was wondering what to expect if I wanted to put my 780, or 7970 on this board, I'm running a really old bios from 4/12 IIRC.


----------



## beaker7

Quote:


> Originally Posted by *anubis1127*
> 
> Has anybody had any luck running newer PCI-E 3.0 GPUs on this board? Does the newer BIOS revisions fix the issues I read about early on? I'm just using the on-board VGA header for display right now, but was wondering what to expect if I wanted to put my 780, or 7970 on this board, I'm running a really old bios from 4/12 IIRC.


Running 7 Titans no problem. 3 internal, 4 in PCI-E breakout box.


----------



## anubis1127

Quote:


> Originally Posted by *beaker7*
> 
> Running 7 Titans no problem. 3 internal, 4 in PCI-E breakout box.


Thanks! Which BIOS rev are you on? What OS?


----------



## TheBlademaster01

psyq said that his B0 chips ran along with a Titan. Just no PCI-e 3.0 speeds. Now he has 2697s and latest BIOS (one after 5004 I believe) and his Titan runs at 3.0.

Since you have C0, I think you should just go for the latest if you want 3.0 speeds. Otherwise, just test the cards


----------



## anubis1127

Thanks, BM!

I've got two plus months uptime on my server right now, not sure I want to take it down, hahaha.

It does need to go in a case at some point, when I do that I may update the bios and throw a GPU in it.


----------



## beaker7

Quote:


> Originally Posted by *anubis1127*
> 
> Thanks! Which BIOS rev are you on? What OS?


bios 3302 according to cpu-z
win7 x64 pro


----------



## anubis1127

Quote:


> Originally Posted by *beaker7*
> 
> bios 3302 according to cpu-z
> win7 x64 pro


Thanks for the info, +rep. You wouldn't happen to have any pics of your build uploaded anywhere? Sounds like an impressive rig.


----------



## TheBlademaster01

It's always those dudes with the anonymous avatars/undercover profiles that have those crazy Titan setups









Maybe if I delete my avatar I'll get some too


----------



## beaker7

Quote:


> Originally Posted by *anubis1127*
> 
> Thanks for the info, +rep. You wouldn't happen to have any pics of your build uploaded anywhere? Sounds like an impressive rig.


Sorry not at the moment. I'll post some of the new Ivy-EP rig when its done though. Here is what it will contain:

Lian Li A75 case
Corsair AX1200i
Supermicro X9DA7 board
128 or 256 GB ECC RAM either 1600 or 1866 mhz (TBD)
2x Ivy-EP CPUs (which model tbd, either the 12 x 2,7 ones or was still hoping for a 10 x 3.4 2687w-v2)
Samsung 840 Pro 512 GB system drive
2x iofx 1.65 TB in RAID-0 for fast data
8x 3TB Seagate 7200 RPM drives in RAID-0 for big data
4x 4TB Seagate 7200 RPM drives in RAID-0 for online backup
1x EVGA Titan SC for primary
1x Intel X540 T2 10 Gb NIC
1x Red Rocket - X
1x Netstor PCI-E 3.0 x16 interface card that goes out to an NA255A that contains 4 Titans for CUDA compute

I have everything already except the memory and CPUs. The Rocket and the IOFX cards are on preorder.


----------



## BlackenedTush

Quote:


> Originally Posted by *beaker7*
> 
> Sorry not at the moment. I'll post some of the new Ivy-EP rig when its done though. Here is what it will contain:
> 
> Lian Li A75 case
> Corsair AX1200i
> Supermicro X9DA7 board
> 128 or 256 GB ECC RAM either 1600 or 1866 mhz (TBD)
> 2x Ivy-EP CPUs (which model tbd, either the 12 x 2,7 ones or was still hoping for a 10 x 3.4 2687w-v2)
> Samsung 840 Pro 512 GB system drive
> 2x iofx 1.65 TB in RAID-0 for fast data
> 8x 3TB Seagate 7200 RPM drives in RAID-0 for big data
> 4x 4TB Seagate 7200 RPM drives in RAID-0 for online backup
> 1x EVGA Titan SC for primary
> 1x Intel X540 T2 10 Gb NIC
> 1x Red Rocket - X
> 1x Netstor PCI-E 3.0 x16 interface card that goes out to an NA255A that contains 4 Titans for CUDA compute
> 
> I have everything already except the memory and CPUs. The Rocket and the IOFX cards are on preorder.


beaker7,

What an incredible and _$$$$_ plan for flagship PCIe IOPs... wow! WOW!... _







_
Did I say WOW? I can't wait to see pics and hear the test results!
Don't tell anyone, but at this point I'd be happy with just one Fusion-IO card!
Are you thinking you need two of them to remove bottlenecks for the Red Rocket and the 7 Titans...?

BT


----------



## EndlessRed

Hi all, new Z9PE-D8 here, and I'm running into two very annoying issues that I'm completely stumped on, hopefully someone has an answer for this before I start marching to ASUS with pitchforks.

Here's my system setup:

Asus Z9PE-D8
2x E5-2670 C0
8x4GB Mushkin Blackline non-ECC
2x Samsung 840 Pro SSD
1x EVGA Geforce 760 Superclocked
1x Corsair AX1200

My setup is quite tame compared to many in this thread, which makes it even more difficult to understand.

First issue - my video card is detected as using PCI-E 2.0, when the 2670 C0 is capable of 3.0, as well as the 760 video card. I'm already on the latest BIOS version (5103 if i recall). No setting I've played with can get it to bump up to 3.0. I know performance gain is supposedly minimal, but it does annoy the hell out of me.

Second (and maybe related) issue - video games I play has stuttering. It stutters every several seconds. I've tried a gtx580, gtx660, and a gtx760. Tried all x16 lanes, and I can't get the stuttering to go away. The same games I play on my old Q9650 with the same cards have no stuttering or delay at all.

I've basically narrowed it down to a problem with the motherboard, but what exactly? Is there something I can do to fix, or should I just hit up ASUS?

*UPDATE:* So I decided to try and run some 3D benchmarks and see if it has stuttering...surprisingly, it doesn't. Besides the standard frame drops in complex rendering, it doesn't exhibit the stuttering I'm seeing in video games. I'm thinking it's also possible that my SSD RAID using onboard RSTe has its own issues, probably would try a separate install on a regular drive and see.


----------



## squidman

Back to the old reliability of hdd thread:

anyone with experience/views on seagate vs WD 4 tb drives? Still looking to populate the last 4 sata ports on the bottom of the board with 4x4tb hdd's...but am very leary of seagate at the moment. Looking at the WD Red series (the cheaper non-enterprise ones), and willing to pay the 100 bucks extra after last catastrophic seagate failure. Basically in raid 10 or 1+0 config. Have never had a problem with WD green either apart from one DOA (prbly shipping damage), which are quite a bit cheaper. Any thoughts/suggestions? Getting a bit paranoid here (thus raid 10 instead of raid 5).

Any dramas with the controller (guess it's the intel controller there/SAS/SATA thing).

Thanks cookie out....wow wish I had 7 titans too!


----------



## BlackenedTush

Quote:


> Originally Posted by *squidman*
> 
> anyone with experience/views on seagate vs WD 4 tb drives? Still looking to populate the last 4 sata ports on the bottom of the board with 4x4tb hdd's...but am very leary of seagate at the moment. Looking at the WD Red series (the cheaper non-enterprise ones), and willing to pay the 100 bucks extra after last catastrophic seagate failure. Basically in raid 10 or 1+0 config. Have never had a problem with WD green either apart from one DOA (prbly shipping damage), which are quite a bit cheaper. Any thoughts/suggestions? Getting a bit paranoid here (thus raid 10 instead of raid 5).


Squidman,

I agree... most peeps seem to favor Westrern Digital slightly over Seagate these days. But I have no clue how to measure. It's mostly hearsay and seems more like luck of the draw to me. I think that if you purchase a decent drive from either you up your chances. I've bought 11 drives in the past year and they have either been Seagate Barracuda, Barracuda XT, WD Black, or WD Green.

On my workstation I have 4x 2TB Seagate Baracuda XTs in RAID 10 (total yield 3.6GB). They're controlled by an Adaptec 6805e card. The purpose of the RAID is storage of critical school & life files and for use as a rendering target... (so RAID 1+0). I did make sure that this Seagate Barracuda XT model was compatible with the card; but I was (and still am) a little paranoid that they weren't _enterprise level drives_ with _Error Recovery Control_ (ERC). Western digital calls it _"Time-Limited Error Recovery"_ (TLER). Anyway, lots of people take their chances just like me because the higher cost of enterprise drives gets multiplied further in RAID. So far so good... the combination has performed flawlessly for me.
But...
we shall see if(when) some bad sectors start cropping up (see "Consumer drives in RAID..." below)

*Mitigating risk is important...*
All that said... I did make sure that my RAID had an identical 5th drive to set as a global hotspare. It automatically kicks in should there be a problem with one of the other 4. But that is not all. I would never set up a storage system for important files (RAIDed or not) that doesn't also have a solid backup scheme... I use Windows Backup to back up my important data plus system images. It goes off nightly and I alternate full and incremental compressed BUs between two 3TB WD greens. They pop into my front hot-swap bays so I can easily cycle them offsite. I also have a 2.5" laptop HDD fitted in an Icy Dock hotswap bay that's reserved for BUs of in-progresss projects, using the "FreeFileSync utility to make things quick & easy to BU... and recover quickly if needed.

*Hard Drives in General...
*Hard drives may have improved somewhat over the years; but because the immense capacity, the MTBF/drive is probably increasing. The bigger the drive, the more chance there is of having a media failure somewhere. The more you can store, the more you can lose. It's amaziing how easily one can "put all his/her eggs into one (fragile) basket" these days! That's why I compromised with 2GB drives.

*Consumer drives in RAID...*
Not having ERC or TLER drives can mess with the RAID's fault tolerance schemes when little read/write problems start to "surface" due to bad sectors. Modern consumer drives are built to respond by flagging the sectors as bad and moving on to another area of the disk. But if this happens during a write to the RAID, it doesn't communicate to the RAID controller that it is doing so. And if this takes too long, the RAID controller could simply see the drive as being unresponsive and assume that it has failed. It will then try to rebuild the array from parity data. With no hotspare, it does so with remaining disks. And if there is another similar failure on another drive during the array rebuild, it could be catastophic. With an ERC drive that has some bad sectors, it will work with the controller to send the data elsewhere before taking the time to flag the sectors... so the drive & the RAID stays in normal operation. Because of the complex parity striping I would never do a RAID 5 without ERC drives. I probably would never find a reason to do RAID 5 actually.
Quote:


> Originally Posted by *squidman*
> 
> ...wow wish I had 7 titans too!


But what would _you_ do with 7 functioning Titans? Assuming you can get them all working... Beaker7 has a Red Rocket card so I assume he's into 444 4k video. But what video application can use that many CUDA cores? Davinci Resolve? Unless you hack them, you can't use the workstation quality drivers to get both screaming _and precise_ 3D...
It's a mystery I am staying tuned for. For me, having 7 Titans would give me the ability to say, "I have 7 Titans"... haha

BT


----------



## squidman

Hey thanks for exhaustive reply svarte ræva! Setter pris på det! appreciated!

Yeah I'm getting a bit paranoid (looking at jotta.no for unlimited backups for 50 nok/10usd/month), but actually uploading multiple terabytes when I'm on 30down/5/up could be quite a challenge!
7 Titans? Octane render (or Furry Ball iff you have the dough!) i.e. gpgpu rendering....still sticking with regular old vray for now and messing with arnold....

Having said that, my lonely gtx 770 4gb (yeah the vram is important for gpgpu still!) takes 2 spaces, so couldn't actually fit 7 double-spaced cards....and mine is one pci-e 4 (black x8) never got it to work on a x16 lane not that I'd notice the difference! Mostly glad it's working at all!

Still have to learn resolve and scratch....wow so mucho still to learn actually (Nuke, Fusion, Houdini, Zbrush, Mari, Softimage, Realflow, lightwave, etc. etc.).

Think for color grading of my fishing/skiing trips for the folks it suffices with a traditional grey card! Always with me but never actually used.....









Addendum to BT...wow, I'm cutting it short at raid 10 with 8tb! That's why raid 5 is so tempting. don't ask me how I need so much storage (I never delete anything, and have years of backups and hd footage! of various trips)....so yeah, even 8tb is really not enough....would be more comfortable with raid 10 and 15-20gb usable....but my corsair 900d has already run out of drive bays and they haven't listed more....corsair really are diabolically bad....









Having a look through various articles on serve the home etc....but losing 3 tb due to a drive suddenly going raw (Seagate goflex that is you! and ps your forum members hate you!) just sucks b....s. That's pretty much why I'm in the WD camp for now, but can't quite justify the enterprise versions....


----------



## squidman

Again wow what a useful reply svarta ræve! Bit of a late night here, so I will read through again in the cold light of morning etc.! Did actually send this off to a coz who is a pretty senior dude in the field, so will hear what he has to say (sent similar question...not this thread). But doubt he can do better than you. I guess it boils down to whether you are on the wrong end of a bad batch (and QC) or not. And how they control that etc. As I understand it, either you'll be ok or you'll be one of 4-600 who receive bad drives out of millions. The bad thing is of course iff they die AFTER quality control.

So metrics; MTBF, ERC/TLER, AFR (Annualized failure rate), there were quite a few on some articles long since forgotten...









God natt/good night cookie out!


----------



## BlackenedTush

Squidman,

For video peeps a solid strategy for online/offline data becomes imperative at some point. Maybe you haven't reached that point. haha

I've been trying to come to my senses about my relationship to my data myself. so much so that my butt's turning white...








And here's a thought that I personally struggle against - most peeps don't need instant live access to everything they have... me included.

Setting up an archive system makes a lot of sense. But I don't wanna forget what I have or lose it in some dark corner somewhere, That's what I associate with the idea of offline data. But I'm conceding that if there's some easy way to catalog any offline data (so I could know what I got and get to it relatively quickly/easily).... that would be the key.

I've seen many storage systems and strategies based on tape & optical disk, with NAS, external RAID, and Drobo enclosures/arrays, and so forth... but also low-tech manual schemes using just bare SATA HDDs. This approach isn't very sexy; but it's pretty common in video, don't you think?. It certainly avoids the expensive overhead of the system/enclosure/controller - probably making bare drive storage the cheapest/easiest temporary or even permanent solution. And with the layer of manual human interaction, there's more protection against viruses, etc.

To make this work, I thot I'd get a caddy like *this*. And I also have a Blu-Ray burner that can make redundant BUs that are small & lightweight enough to make offsite archiving easier.

Hesre"s a guy with a decent *grass-roots method of archiving stuff* using bare drives. I hadn't seen the drive cases that he uses. Pretty cool. And he has a fireproof safe to hold it all, which I have been threatening to buy forever...

But again for me, I really feel the need to have a solid connection with, and access to, all of my stuff... maybe that's you 2.
A good catalog system would be the deciding factor between keeping stuff online (and busting at the seams), and moving it offline to free up space. I'll have to see what's out there.

BT


----------



## BlackenedTush

Quote:


> Originally Posted by *EndlessRed*
> 
> Hi all, new Z9PE-D8 here, and I'm running into two very annoying issues that I'm completely stumped on, hopefully someone has an answer for this before I start marching to ASUS with pitchforks.


EndlessRed,

Haha! Probably most of us have been there!
Quote:


> Originally Posted by *EndlessRed*
> 
> First issue - my video card is detected as using PCI-E 2.0, when the 2670 C0 is capable of 3.0, as well as the 760 video card. I'm already on the latest BIOS version (5103 if i recall). No setting I've played with can get it to bump up to 3.0. I know performance gain is supposedly minimal, but it does annoy the hell out of me.
> 
> Second (and maybe related) issue - video games I play has stuttering. It stutters every several seconds. I've tried a gtx580, gtx660, and a gtx760. Tried all x16 lanes, and I can't get the stuttering to go away. The same games I play on my old Q9650 with the same cards have no stuttering or delay at all.
> 
> I've basically narrowed it down to a problem with the motherboard, but what exactly? Is there something I can do to fix, or should I just hit up ASUS?
> 
> *UPDATE:* So I decided to try and run some 3D benchmarks and see if it has stuttering...surprisingly, it doesn't. Besides the standard frame drops in complex rendering, it doesn't exhibit the stuttering I'm seeing in video games. I'm thinking it's also possible that my SSD RAID using onboard RSTe has its own issues, probably would try a separate install on a regular drive and see.


You'd wanna avoid the dreaded RMA process if you can help it.
I'm not an expert, but looking at the common denominators, it might not hurt to check the following:

I always advise this... make sure your CPU coolers aren't mounted too tightly. Some cooler designs can distort the CPU seats and have been known on this mobo to cause intermittent PCIe graphics issues.
If you have the NVIDIA control panel installed and if the Verticle Sync set as "adaptive"... (or a compatible setting)
Latest NVIDIA drivers.
Make sure your GPUs are getting enough juice from your PSU. I see you have an EVFA GPU... maybe use Precision X to check how much voltage is getting to it. Maybe a PSU or power cable glitch.
PCIe Subsystem setting in BIOS that someone might have inadvertently tweaked... I don't know, like maybe _PCI Latency Timer_? I know that _Palette Snoop_ has been known to affect NVIDIA cards on this mobo (maybe not in that way...). I'm reachin... Manual tells default setting.
Make sure the two PCIe jumpers on the board itself haven't been tweaked (refer to manual):
[SMBUS connection setting jumper by slot 4 (TESLA_M_SW) & the VGA controller setting (DIAG_VIEW1)]
I've heard of hard drives being replaced by SSDs solving stuttering; but you have SSDs

Good luck!

BT


----------



## EndlessRed

Quote:


> Originally Posted by *BlackenedTush*
> 
> You'd wanna avoid the dreaded RMA process if you can help it.
> I'm not an expert, but looking at the common denominators, it might not hurt to check the following:
> 
> I always advise this... make sure your CPU coolers aren't mounted too tightly. Some cooler designs can distort the CPU seats and have been known on this mobo to cause intermittent PCIe graphics issues.
> If you have the NVIDIA control panel installed and if the Verticle Sync set as "adaptive"... (or a compatible setting)
> Latest NVIDIA drivers.
> Make sure your GPUs are getting enough juice from your PSU. I see you have an EVFA GPU... maybe use Precision X to check how much voltage is getting to it. Maybe a PSU or power cable glitch.
> PCIe Subsystem setting in BIOS that someone might have inadvertently tweaked... I don't know, like maybe _PCI Latency Timer_? I know that _Palette Snoop_ has been known to affect NVIDIA cards on this mobo (maybe not in that way...). I'm reachin... Manual tells default setting.
> Make sure the two PCIe jumpers on the board itself haven't been tweaked (refer to manual):
> [SMBUS connection setting jumper by slot 4 (TESLA_M_SW) & the VGA controller setting (DIAG_VIEW1)]
> I've heard of hard drives being replaced by SSDs solving stuttering; but you have SSDs
> 
> Good luck!
> 
> BT


Thanks for the tips m8, but no luck...tried everything from loosening the heatsink (or pump in my case), plenty of juice from the PSU, jumpers aren't out of place...yeah I think I might have to RMA my board as well







if it wasn't for my video editing/encoding i would just go to a Ivy Bridge-E i7 on a more reliable mobo...urgh.

I remember last time I RMA'd something from ASUS, it took a month and a half







: hope they can do advance replacement instead.


----------



## ignoramus

So she's finally built! I still have some setting up to do ie. ironing out the multiple restarts, etc but shes crunching 10 times more than my laptop (M1730) and 3 times more than my previous FX8150 rig!

Check it out:


+Rep to the blademaster for answering all of my nooby questions - it was a great help!

and how do i go about joining this forums folding team? i figure you guys and this thread helped me out so ill give back by folding on your team.







(plus ill be living in bills included accommodation for 2 years so you can expect 24/7 folding!)


----------



## TheBlademaster01

Looking good ignoramus







. And yes, I know what you mean by getting a boost. It's especially the bandwidth of this system that really helps me. A lot less slowdowns while working on large scenes, doing multiple things at once etc.

You can join the folding team by just entering 37726 as folding team. Usually people run bigadv on their 2P rigs and join the Coremageddon. But you need to run a Linux VM or native to get them.

Look here if you're interested









http://www.overclock.net/t/1222125/coremageddon-bigadv-lounge/0_30


----------



## EndlessRed

Quote:


> Originally Posted by *ignoramus*
> 
> So she's finally built! I still have some setting up to do ie. ironing out the multiple restarts, etc but shes crunching 10 times more than my laptop (M1730) and 3 times more than my previous FX8150 rig!
> 
> Check it out:
> 
> 
> +Rep to the blademaster for answering all of my nooby questions - it was a great help!
> 
> and how do i go about joining this forums folding team? i figure you guys and this thread helped me out so ill give back by folding on your team.
> 
> 
> 
> 
> 
> 
> 
> (plus ill be living in bills included accommodation for 2 years so you can expect 24/7 folding!)


Does your gpu get detected as PCI-E 3.0?


----------



## Marchitect

Quote:


> Originally Posted by *EndlessRed*
> 
> Hi all, new Z9PE-D8 here, and I'm running into two very annoying issues that I'm completely stumped on, hopefully someone has an answer for this before I start marching to ASUS with pitchforks.
> 
> Here's my system setup:
> 
> Asus Z9PE-D8
> 2x E5-2670 C0
> 8x4GB Mushkin Blackline non-ECC
> 2x Samsung 840 Pro SSD
> 1x EVGA Geforce 760 Superclocked
> 1x Corsair AX1200
> 
> My setup is quite tame compared to many in this thread, which makes it even more difficult to understand.
> 
> First issue - my video card is detected as using PCI-E 2.0, when the 2670 C0 is capable of 3.0, as well as the 760 video card. I'm already on the latest BIOS version (5103 if i recall). No setting I've played with can get it to bump up to 3.0. I know performance gain is supposedly minimal, but it does annoy the hell out of me.
> 
> Second (and maybe related) issue - video games I play has stuttering. It stutters every several seconds. I've tried a gtx580, gtx660, and a gtx760. Tried all x16 lanes, and I can't get the stuttering to go away. The same games I play on my old Q9650 with the same cards have no stuttering or delay at all.
> 
> I've basically narrowed it down to a problem with the motherboard, but what exactly? Is there something I can do to fix, or should I just hit up ASUS?
> 
> *UPDATE:* So I decided to try and run some 3D benchmarks and see if it has stuttering...surprisingly, it doesn't. Besides the standard frame drops in complex rendering, it doesn't exhibit the stuttering I'm seeing in video games. I'm thinking it's also possible that my SSD RAID using onboard RSTe has its own issues, probably would try a separate install on a regular drive and see.


Someone on this thread posted a link for 'forcing' your Nvidia graphics cards to run in 3.0 mode.
It's a link from Nvidia and after you run the program patch, your cards will most likely run in 3.0 mode but you will only notice this if you run something like CPU-z.
I've tried this and did not get any better results in Cinebench 11.
My general advice would be to stick with 2.0, especially if your machine is stable. Don't expect anything special after running this patch from Nvidia.


----------



## Marchitect

Quote:


> Originally Posted by *Marchitect*
> 
> Someone on this thread posted a link for 'forcing' your Nvidia graphics cards to run in 3.0 mode.
> It's a link from Nvidia and after you run the program patch, your cards will most likely run in 3.0 mode but you will only notice this if you run something like CPU-z.
> I've tried this and did not get any better results in Cinebench 11.
> My general advice would be to stick with 2.0, especially if your machine is stable. Don't expect anything special after running this patch from Nvidia.


Here is the link I am referring to:

http://nvidia.custhelp.com/app/answers/detail/a_id/3135/~/geforce-600-series-gen3-support-on-x79-platform


----------



## ignoramus

Quote:


> Originally Posted by *EndlessRed*
> 
> Does your gpu get detected as PCI-E 3.0?


I found out through driver genius (which gives much better information than speccy) that im not :/ how do i go about rectifying that?

Do we know what causes the multiple restarts? waking from sleep works flawlessly...

Last question. Do i dare upgrade the bios to 3509 by first updating to 3109? Will there be no benefit ie memory and cpu working that bit more efficiently? i cant help but feel at 2.5ghz i should be getting 20+ in cinebench when 2650s can do better...


also ill eventually get around to updating all the drivers, its just not that strait forward when youre on a *cough student budget *cough


----------



## TheBlademaster01

I went through that same phase lol. Between 18 and 19 is okay for B0 2665s. C0 or retail 2665s boost up about 300 or 400MHz higher. Also, you need the latest BIOS version to get PCI-e 3.0 support, so far only psyq's 2697 are running that. B0 can't go further than 3509 as you know.


----------



## beaker7

Retail chips are out. the 2690's look like the sweet spot. Disappointed the 2687's are still only 8 core.


----------



## BlackenedTush

Quote:


> Originally Posted by *beaker7*
> 
> Retail chips are out. the 2690's look like the sweet spot. Disappointed the 2687's are still only 8 core.


Beaker7,

It had been my understanding that the 2687s had attributes that made them better suited for workstations, while the 2690's were better for servers... Was I wrong? Or is it just a new playing field now given Intel's latest core/clocking strategy?

BT


----------



## anubis1127

I installed W7 on my Z9PE-D8 WS board today.









Mmmmm, threads:


----------



## EndlessRed

*UPDATE:* So, while I'm waiting for an advanced replacement RMA from ASUS (yeah I called it in







), I decided to continue digging for possible causes to the stuttering. After taking a closer look at CPU-Z, I noticed one of the CPUs is stuck at a minimum of 26x multiplier, while the other one is at x12/24/26/whatever. I have both SpeedStep and Turbo mode enabled, so I became curious and disabled both turbo mode and SpeedStep, then set the multiplier at a static 26x, so both CPUs run at 26x multiplier regardless of any situation.

The stuttering is gone.

WHAT THE DEUCE!? So I think, based on this, it's conclusive that there's a problem with my board. I know the CPUs aren't supposed to run literally in parallel, but even on hours of idle, the other CPU was stuck at a 26x multiplier.

So...since I have to return the motherboard anyway...I think I'll attempt to overclock it to kingdom come just to vent out my frustration









EDIT: Yeah, forget about the overclocking...the whole system freaks out at just BCLK of 105 lol


----------



## ZDngrfld

Quote:


> Originally Posted by *EndlessRed*
> 
> *UPDATE:* So, while I'm waiting for an advanced replacement RMA from ASUS (yeah I called it in
> 
> 
> 
> 
> 
> 
> 
> ), I decided to continue digging for possible causes to the stuttering. After taking a closer look at CPU-Z, I noticed one of the CPUs is stuck at a minimum of 26x multiplier, while the other one is at x12/24/26/whatever. I have both SpeedStep and Turbo mode enabled, so I became curious and disabled both turbo mode and SpeedStep, then set the multiplier at a static 26x, so both CPUs run at 26x multiplier regardless of any situation.
> 
> The stuttering is gone.
> 
> WHAT THE DEUCE!? So I think, based on this, it's conclusive that there's a problem with my board. I know the CPUs aren't supposed to run literally in parallel, but even on hours of idle, the other CPU was stuck at a 26x multiplier.
> 
> So...since I have to return the motherboard anyway...I think I'll attempt to overclock it to kingdom come just to vent out my frustration
> 
> 
> 
> 
> 
> 
> 
> 
> 
> EDIT: Yeah, forget about the overclocking...the whole system freaks out at just BCLK of 105 lol


You'll hit 104 as a max on pretty much every CPU. They locked these E5s down something fierce.


----------



## beaker7

Quote:


> Originally Posted by *BlackenedTush*
> 
> Beaker7,
> 
> It had been my understanding that the 2687s had attributes that made them better suited for workstations, while the 2690's were better for servers... Was I wrong? Or is it just a new playing field now given Intel's latest core/clocking strategy?
> 
> BT


There isnt any special sauce in the E5-2687w v2's to my knowledge that make them particularly suited to workstation. Just being marketed that way due to their higher core speed at the expense of number of cores.

Most of my work is well multithreaded though so 10x3.0 may be a better overall balance than 8 x 3.4 or 12 x 2.7


----------



## beaker7

So what do you guys think:

Crucial
Part Number: CT2K16G3ERSDD4186D
Module Size: 32GB Kit (16GBx2)
Package: 240-pin DIMM
Feature: DDR3 PC3-14900
Specs: DDR3 PC3-14900 • CL=13 • Dual Ranked • x4 based • Registered • ECC • DDR3-1866 • 1.5V • 2048Meg x 72 •

or

Crucail
Part Number: CT2K16G3ERSLD4160B
Module Size: 32GB Kit (16GBx2)
Package: 240-pin DIMM
Feature: DDR3 PC3-12800
Specs: DDR3 PC3-12800 • CL=11 • Dual Ranked • x4 based • Registered • ECC • DDR3-1600 • 1.35V • 2048Meg x 72 •

or

Kingston ValueRAM
Model KVR16R11D4K4/64I
Type 240-Pin DDR3 SDRAM
DDR3 1600
Cas Latency 11
Voltage 1.5V
ECC Registered
With Thermal Sensor


----------



## ignoramus

12 core IVB-E B1 stepping for a cool £800

here!


----------



## PCModderMike

Quote:


> Originally Posted by *anubis1127*
> 
> I installed W7 on my Z9PE-D8 WS board today.
> 
> 
> 
> 
> 
> 
> 
> 
> 
> Mmmmm, threads:


----------



## TheBlademaster01

You really appear in unexpected places, Mike









@ignoramus

It's not really that great of a price for a B1 chip. I've seen 2695 B1 on Taobao for 500 pounds a piece


----------



## PCModderMike

Quote:


> Originally Posted by *TheBlademaster01*
> 
> You really appear in unexpected places, Mike


Cruising OCN


----------



## squidman

Edit:reply to blackened tush.....lost the quote somehow...

Yeah I have two external usb 3.0 caddies like your link and now of course 4? seagate attachments which are basically the same thing. Think I brought up this paranoia on dvxuser or dviinfo.net at some point back in the hdv days (now 2! shoe boxes with the good old hdv tapes! at least 200!), and I recall one dude saying how he backed up to external, put them in a safety deposit box, then rotated them etc.!







Ok not the most practical method (wow missing those hdv tapes! at least you had a good backup!).

Spoke to our norwegian online heroes...jotta.no (who probably made a mint out of the nsa scandal!), very reasonable rates (around 120/year for unlimited backup), but have to say I would almost rather the nsa look at my old home-vids than take the various risks going to a smaller company entails...(operational risk, finance risk, etc etc for you risk-averse people out there)...but especially the risk of them going bust. I guess as risk-proof you could be would be to store with amazon s3 but their rates are pretty prohibitive (also for rendering! but can be done!).

As a matter of fact looking at some of the old hv20/hv30 hdv tapes as we speak! Albeit on a tiny 28" hdtv, but, they don't look too bad! Very video, and highlights blown out to hell, but pretty sharp considering 1440p. Not sure iff my smartphone takes better footage, lol! Helped that back then a tripod was mandatory! These days it's shoot away and hope mercalli or AE can stabilize...(my back appreciates it tho!)...


----------



## squidman

For those rendering on Vue Xstream or Vue...yeah it helps with this mobo!

My times rendering the lake tahoe Vue scene from the extra's disc to 1080p at broadcast quality...

1) lappie: core i7-2630qm with nvidia gt 540m 2gb and sloow 5400 rpm hdd...
2) the 'obelisk': 900d with our asus ws inside and 2x B0 stepping e5-2665 ES (all on ssd of course!)..

lappie time: 5h03'
obelisktime: 33'

Yeah I'm stoked! Fumefx and Mercalli also rock! (ok coming from a VEERY low starting point! lol!)! Mercalli now an hour instead of days!

Still not realtime, but I can live with that for now







GOD I love that beast of a machine! And despite Corsair being absolutely diabolical in software, firmware, etc etc especially customer support...that 900d is just ripping!


----------



## EndlessRed

Quote:


> Originally Posted by *ZDngrfld*
> 
> You'll hit 104 as a max on pretty much every CPU. They locked these E5s down something fierce.


No kidding...? What a shame. I'm betting these E5's got some mad potential to them.


----------



## TheBlademaster01

Definitely. Not sure about the boards though.


----------



## ZDngrfld

Quote:


> Originally Posted by *EndlessRed*
> 
> No kidding...? What a shame. I'm betting these E5's got some mad potential to them.


Some people have hit a whopping 105!!! It's really rare and for the most part moderately unstable. It seems like the people hitting 105 end up having issues down the road and back down to 104 for stability.


----------



## anubis1127

I can get up to 107 on a single e5, xD


----------



## TheBlademaster01

Yeah, it's the boards most likely. Just like the dual boards have issues running 1600MHz+


----------



## EndlessRed

Quote:


> Originally Posted by *anubis1127*
> 
> I can get up to 107 on a single e5, xD


WITCHCRAFT!!!


----------



## ignoramus

Quote:


> Originally Posted by *TheBlademaster01*
> 
> @ignoramus
> 
> It's not really that great of a price for a B1 chip. I've seen 2695 B1 on Taobao for 500 pounds a piece


Way too expensive for me at the moment anyway.. I just looked and it turns out my mobo was shipped with BIOS version 3302, any reason why i shouldn't update to 3506? Im curious why you are staying on 0703...


----------



## TheBlademaster01

I'm on 0503 even. I'm staying with it because it works and I don't want to brick anything


----------



## anubis1127

Quote:


> Originally Posted by *TheBlademaster01*
> 
> I'm on 0503 even. I'm staying with it because it works and I don't want to brick anything


I am on 0503 too.


----------



## BlackenedTush

Quote:


> Originally Posted by *anubis1127*
> 
> I am on 0503 too.


Me 2. Old school.


----------



## anubis1127

Quote:


> Originally Posted by *BlackenedTush*
> 
> Me 2. Old school.


I'm definitely in the "If it isn't broken, why fix it" camp when it comes to BIOS. Well most things really.


----------



## BlackenedTush

Quote:


> Originally Posted by *squidman*
> 
> Spoke to our norwegian online heroes...jotta.no (who probably made a mint out of the nsa scandal!), very reasonable rates (around 120/year for unlimited backup), but have to say I would almost rather the nsa look at my old home-vids than take the various risks going to a smaller company entails...(operational risk, finance risk, etc etc for you risk-averse people out there)...but especially the risk of them going bust. I guess as risk-proof you could be would be to store with amazon s3 but their rates are pretty prohibitive (also for rendering! but can be done!).


I've never been a fan of cloud storage, altho the heroic _Jottocloud_ is intriguing - not so much for the protection from agencies of the US patriot Act (big brother NSA isn't the only snoop on the internet with exploitative intentions for our personal data). It's a big journey with a myriad of vulnerable hubs & switchers between Arizona & Oslo... Actually I just had to check. I just did a trace-route from Phoenix to Jotto. There are 20 or so hops. I'm intrigued with Jotto because it's _relatively_ close (pun intended) to where my family roots are in Strandbygda & Elverum! So If my beloved "digitals" ever did get through the gauntlet of those 20 vulnerable nodes unscathed, it would at least feel like it was _"home safe"_









Quote:


> Originally Posted by *squidman*
> 
> These days it's shoot away and hope mercalli or AE can stabilize...(my back appreciates it tho!)...


Never heard of Mercalli... I'll have to check it out. Would you recommend the regular version or the v2 Pro version?


----------



## BlackenedTush

Quote:


> Originally Posted by *anubis1127*
> 
> I'm definitely in the "If it isn't broken, why fix it" camp when it comes to BIOS. Well most things really.


Exactly


----------



## squidman

Yes prbly lots of hops, not worth it iff you're already based in the US! But then even jotta.no go through sweden, who pass on all details (and record them!) to big brother. Actually looked for a bit at various Bahamas etc based services, but as am not into chil.-P...., well the nsa etc can look into my files for all I care. May still go there just on principle! We seem to be moving towards some kind of scary big brother world regardless...just really really going the wrong way...!! Yeah us europeans and the rest of the world aren't too too happy!!














But as most of the governments have been accomplices here, weeeelll, what are you gonna do? I accepted that I didn't have private info long ago once I signed onto facebook...and already then accepted that everything via google etc would be open to the 'authoritas'...sic...'respect my authorita!'

Our only hope as private citizens is that they drown in information...


----------



## squidman

Svarta ræva,

Mercalli is just stabilizing software...look up prodad...much faster and better than adobe version...but pricey....both plugins for AE and stand-alone available...not convinced so far with their gopro version....åå

never mind...I usually just run all my samsung mobile vids which are hand-held through the stand-alone, but it doesn't let you save project files...lose quite a bit of resolution at say 20-40% on bog-standard settings, but at 10% it's not too bad. Given that it's a mobile h.264 file to begin with...

In the stand-alone, you can also only save to h264 in low, medium, or high quality..no idea about bitrate. But the stand-alone is sooooo!!! much faster than the plugin in AE or AE's own warp stabilizer. Of course for clients, maybe go the warp stabilizer...but for casual things, mercalli all the way! The beast told me so (weell 32 threads told me so!)...


----------



## squidman

"I am on 0503 too.







"

me too! Still on older Bios! Flashed to the newest that would support ES/spicy chips (god I love ebay and the PRC!) but then left it well alone! On some dodgy B0 stepping here, but damn they actually work! so chuffed you have no idea!


----------



## Arkuatic

Anyone care to help me figure this out? Really don't want to pay $45 to RMA to board to asus.

Here is the issue: The motherboard does not post, boot, or even power on for more than a split second.

Originally the motherboard worked fine until I upgraded the bios then it stopped getting to the bios. Attempted crashfree bios etc to no avail. and at one point the motherboard just stopped booting. I purchased a bios replacement from ebay and requested the version to be the same one as I had previously but the same exact problem persists of no boot.

Here's the kicker, if CPU #2's socket is loaded but the 8pin for it isnt plugged in then it powers on all the fans(as if it was properly booting) when the power button is hit. Of course this doesn't allow me to do much since it simply shows the components are being powered but nothing is working.

There isn't any case that it's on except a glass surface so I can't say it's being shorted or at least visually. It's been cleaned and well air canned and is still an impeccable board visually. Aside from CPU, GPU, PSU, and memory there are no other components connected to the motherboard, even keyboard and mice.
Processors are e5.2665's.

Here's what I've tried:

Changed out memory: used adata, and various corsairs; tried in different slots etc. note: I only run either 1 or 2 sticks at a time.

Ran with CPU 1 only(the board can run off one cpu) and swapped processors.

Swapped out different power supply: corsair ax1200i, corsair ax850, and cooler master 850w.

Tried, gtx 580, gtx titan, gt 240, gtx 210.

Rewards One game from my AMD gold code.

Thanks.


----------



## TheBlademaster01

What BIOS versions do not post?


----------



## lowfat

Tried to update to the newest bios last night. Getting code 14 now on boot.









Think ill try grabbing a replacement bios chip from eBay before RMAing it for the second time.


----------



## TheBlademaster01

What is the Q Spec on the Xeons (Qxxx code).


----------



## squidman

With ES E5-2665's B0 stepping, I'm on Bios 3506...apparently the last one that works for B0 stepping. I upgraded from the first (working! BIOS whatever that was) but made sure I was careful to follow the instructions on upgrading to the intermediate bios (see the asus website and this thread). One of the BIOS on the way didn't work at all, thought I'd bricked it. The other things that caused me lots of headaches was not seating the ram properly (man they really have to be in there properly!) and tightening down the screws too much on my cpu coolers (2x corsair h80i's).

Good luck! It's a great board but temperamental! And yes can confirm those ES e5-2665's do work! and BOY for the money can they crunch through some data or rendering, all for the price of one i7! Still it took me about 2 weeks to get the thing half-way functional (figuring out which pci-e slot worked for my 770gtx, getting the friggin' controllers to work in raid, etc!). Almost there now, adding the last 4 hdd's on top of whatever it is the 11 that are there already!

On QAX..'s (8?) here (forget the number). To be honest it sounds like a bad board. Which controller are you booting off? I had to give up on raid 0 on the top two sata slots with samsung 840 ssd's....LSI or whatever it is doesn't like those (although iff you're not posting moot point!). I was also unable to post until I finally found a pcie slot that worked (number 5 black one).

Good luck!


----------



## squidman

Question about power and number hdd's...since I'm here!

So...actually got the extra hdd cages which hold an extra 6hdd with backplanes from Corsair for the 900d (wow they were quick too! under a week!), but now wondering JUST how many (hdd) I can chuck in there without blowing up the power supply.

PS: Corsair 1200 AX
Geforce 770 GTX 4GB Zotac (watts? 350? at load?)
Mobo: z9pe-d8 WS...(with 2x e5-2665's...what around 115 watts each?)..total around 300 watts?
SSD's: 4x (watts? can't be much!)
HDD: 7 so far with another 4 coming and then maybe another 2 (say around 13-14 total)..again wattage, no idea...
Fans: 2x h80i's, 3 in front and one rear exhaust (watts? no idea)
Misc: Corsair commander and cooling link (cooling link powers the 4 above off of one molex)

So is 1200 watts enough to go on? Would think so, although I'm running out of SATA power connections, will be some creative cable routing...









Or do I chuck in a 2nd PS (which would be bad for airflow) to be safe? To power the hdd's (have room for 18 in total).


----------



## beaker7

Quote:


> Originally Posted by *squidman*
> 
> Question about power and number hdd's...since I'm here!
> 
> So...actually got the extra hdd cages which hold an extra 6hdd with backplanes from Corsair for the 900d (wow they were quick too! under a week!), but now wondering JUST how many (hdd) I can chuck in there without blowing up the power supply.
> 
> PS: Corsair 1200 AX
> Geforce 770 GTX 4GB Zotac (watts? 350? at load?)
> Mobo: z9pe-d8 WS...(with 2x e5-2665's...what around 115 watts each?)..total around 300 watts?
> SSD's: 4x (watts? can't be much!)
> HDD: 7 so far with another 4 coming and then maybe another 2 (say around 13-14 total)..again wattage, no idea...
> Fans: 2x h80i's, 3 in front and one rear exhaust (watts? no idea)
> Misc: Corsair commander and cooling link (cooling link powers the 4 above off of one molex)
> 
> So is 1200 watts enough to go on? Would think so, although I'm running out of SATA power connections, will be some creative cable routing...
> 
> 
> 
> 
> 
> 
> 
> 
> 
> Or do I chuck in a 2nd PS (which would be bad for airflow) to be safe? To power the hdd's (have room for 18 in total).


Without even doing the math, you can run a heck of a lot more than that off an AX1200. I have.


----------



## lowfat

Quote:


> Originally Posted by *TheBlademaster01*
> 
> What is the Q Spec on the Xeons (Qxxx code).


I have no idea.









Do I have to run w/ an older bios for certain ES CPUs?


----------



## squidman

Apart from the specs on ebay, where I got mine, here is one place you can find the QXXX numbers and stepping on your cpu's...

http://www.cpu-world.com/sspec/QA/QA8X.html and yes they're very sensitive to the bios, especially upgrading from the bios that came with your mobo, to the intermediate one mentioned by asus, to the latest supported etc. Like a lot of peps mentioned, older is better! when it comes to bios for this mobo. But at least you can revert back with a bios chip as you mentioned. Have yet to go there, fingers crossed!









Edit: I looked at my bios on cpu-z and aida64, but will have to reboot to double check...I think I'm on the same Bios as the earlier posters...i.e. the last one that is good for B0 stepping QA8X...ES chips. Maybe someone can confirm.

Edit:

OK my mobo started on 0503, I flashed to 3109 (the necessary intermediate Bios) and it went pear-shaped. Then flashed to 3506 and all working again, and leaving it well enough alone! Would frankly stick with 0503 tho'! Don't blame me if you brick your mobo following this advice! As far as I know, 3506 is the last Bios that supports early ES chips, but I stand to be corrected. All working on it here except for my stupid H80i's (internal usb headers not initializing fast enough, have to manually plug/replug sata power to them every time).

Edit: more likely crap corsair products, software, firmware, etc (wow the corsair link software is absolute garbage!). At least order overseas came quickly...only redeeming feature (and kick-a$$ 900d case!).


----------



## squidman

Oh yeah..the one thing that really stumped with this board, seating the ram...it really has to be in there properly! Took almost 3 days to figure that one out! But am on the cheapest corsair vengeance ram (4x8gb sticks, all on the left side of cpu's as indicated in the manual). Good luck!









Dunno iff I'm replying to lowfat or arkuatic here, but as long as someone finds it useful...


----------



## TheBlademaster01

Quote:


> Originally Posted by *lowfat*
> 
> I have no idea.
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> Do I have to run w/ an older bios for certain ES CPUs?


What squidman said, 3506 is the last version that works with QAxx CPUs (ES1 B0 chips). 5004 adds Ivy support though.


----------



## anubis1127

This is annoying:



I don't even know where I would get the part needed to fix that.


----------



## TheBlademaster01

What happened?


----------



## anubis1127

Quote:


> Originally Posted by *TheBlademaster01*
> 
> What happened?


That is just how Asus sent it to me, with it missing the screw hole.


----------



## TheBlademaster01

Quote:


> Originally Posted by *anubis1127*
> 
> Quote:
> 
> 
> 
> Originally Posted by *TheBlademaster01*
> 
> What happened?
> 
> 
> 
> That is just how Asus sent it to me, with it missing the screw hole.
Click to expand...

Yeah, I don't really know









Is it the only part without the screw hole?


----------



## lowfat

Quote:


> Originally Posted by *TheBlademaster01*
> 
> What squidman said, 3506 is the last version that works with QAxx CPUs (ES1 B0 chips). 5004 adds Ivy support though.


]
Thanks. I have a couple of i7 3930ks. Will I be able to use it to flash the bios or no?


----------



## anubis1127

Quote:


> Originally Posted by *TheBlademaster01*
> 
> Yeah, I don't really know
> 
> 
> 
> 
> 
> 
> 
> 
> 
> Is it the only part without the screw hole?


Yes. It's just annoying because I can't mount the CPU coolers I bought for it. I have been able to use an old h80, but it still doesn't make great contact without being able to use all 4 screws to get even pressure. I'm thinking about trying one of the Antec CLC (or any with the circle style pump/block) as they mount slightly differently, and maybe that will help.


----------



## anubis1127

Quote:


> Originally Posted by *lowfat*
> 
> ]
> Thanks. I have a couple of i7 3930ks. Will I be able to use it to flash the bios or no?


You can run one i7 3930k I believe, never tried it, but it should work.

Two will not because 3930k is QPI gimped.

Yes I know double post, haha.


----------



## TheBlademaster01

Quote:


> Originally Posted by *lowfat*
> 
> Quote:
> 
> 
> 
> Originally Posted by *TheBlademaster01*
> 
> What squidman said, 3506 is the last version that works with QAxx CPUs (ES1 B0 chips). 5004 adds Ivy support though.
> 
> 
> 
> ]
> Thanks. I have a couple of i7 3930ks. Will I be able to use it to flash the bios or no?
Click to expand...

Probably, I think the system should post with a single 3930K in slot 1 (that's the one furthest away from the PCI-e slots and I/O connectors)
Quote:


> Originally Posted by *anubis1127*
> 
> Quote:
> 
> 
> 
> Originally Posted by *TheBlademaster01*
> 
> Yeah, I don't really know
> 
> 
> 
> 
> 
> 
> 
> 
> 
> Is it the only part without the screw hole?
> 
> 
> 
> Yes. It's just annoying because I can't mount the CPU coolers I bought for it. I have been able to use an old h80, but it still doesn't make great contact without being able to use all 4 screws to get even pressure. I'm thinking about trying one of the Antec CLC (or any with the circle style pump/block) as they mount slightly differently, and maybe that will help.
Click to expand...

Yeah, RMA'ing may give you a bricked board though









Maybe ask them about it?


----------



## anubis1127

Quote:


> Originally Posted by *TheBlademaster01*
> 
> Yeah, RMA'ing may give you a bricked board though
> 
> 
> 
> 
> 
> 
> 
> 
> 
> Maybe ask them about it?


I emailed them this morning, we'll see. I can just deal with it, but it's annoying. I wouldn't care if it were a $150 z77 board, but it was a bit more than that.


----------



## TheBlademaster01

Low quality check IMO


----------



## poulk

Hello guys!
I plan make video editing station on 2P xeon ivy-ep, and want to use Asus Z9PE-D8. And of course I want the maximum overclock these processors. The system will be cooled with water (CPU and motherboard). But now I'm having doubts about the Asus. Will she be able to unleash the full potential of the CPU? The system will be cooled with water (CPU and motherboard). But now I'm having doubts about the Asus. Will she be able to unleash the full potential of the CPU? whether it can change the CPU multiplier (within the boundaries set by Intel), and how many active nuclei can activate it at the maximum multiplier? Can I adjust the BCLK frequency of motherboard? Or maybe supermicro x9dax be much better choice for overclocking?
And as i understand(after fast readint of this topic







) Asus Z9PE-D8 is not very stable motherbord which have not very good quality? or i'm not right?
I would be grateful for any information.


----------



## TheBlademaster01

The board isn't the most stable board but is one of the only ones that offer overclocking on memory and processor. Multiplier overclocking won't do anything, only bus.

If you get an Asus board, get one used from a user who knows it works or try your luck









You won't get much out of the processors either way (104MHz max, vs 100MHz stock).


----------



## poulk

thanks for the reply.
unfortunately the opportunity to buy a used motherboard is not real.
what about Supermicro? maybe she's stable. yes and judging by intrnet it may more than 104 on the BCLK


for 24 cores it's may have some an advantage.
Quote:


> Multiplier overclocking won't do anything, only bus.


I mean can the board to get to work all 12 cores at max multiplier Turbocor available processor (this also applies to the Supermicro)


----------



## DizZz

Multiplier is locked on ivy bridge xeons so you can only overclock the base clock from 100 up to 104 which is only a minimal increase. This Asus board allows you to run memory at 2133mhz though unlike supermicro boards but not everyone has reported success with this. The quality is generally very high although there are a few exceptions. This board has a ton of features so there are more possible things that can go wrong.


----------



## anubis1127

Quote:


> Originally Posted by *poulk*
> 
> thanks for the reply.
> unfortunately the opportunity to buy a used motherboard is not real.
> *what about Supermicro? maybe she's stable. yes and judging by intrnet it may more than 104 on the BCLK*


With one CPU on my 2680s I can OC to around 107Mhz, but with 2 in my z9 board I can't go over 104 with sustained stability.


----------



## TheBlademaster01

Oh yeah the board can do that. But that 107MHz bus frequency is amazing









Maybe it's a quirk of this board that it only does 104MHz

If you don't need the PCI-e lanes and SLI/CF, the Supermicro will be a better choice.


----------



## squidman

Back again with more HDD questions!

Has anyone used a highpoint rocketraid or other raid card without it conflicting with the existing intel/LSI/Marvell chips in the z9pe-d8 WS?

At the moment using the Marvell controller for the 4 grey SATA III slots in 'hyper-duo' mode (their version of IRST I guess - basically 2xsmall ssd's paired with 2x WD Green for smart-caching), the rest are JBOD. Have 4x4TB WD Red NAS drives on the way which are going in the last 4 SAS/SATA slots on the bottom in hopefully raid 1+0/10 (brute force method!). Just don't trust raid 5 at the moment.

So I still have 5xhdd's to whack in there in JBOD, and as I've run out of SATA ports, need a controller. But most of the forums I've looked at mention that as soon as you stick in a raid/jbod card with the same controller, you most often have to disable the mobo controller...not good! So you see my dilemma.

Any suggestions? Need at least 4 sata II ports, but not averse to 8 if it can be done budget (read under 150 bucks). Not really averse to software raid either, in fact would prefer it, just need the extra 4-8 sata ports.

Thanks cookie out


----------



## poulk

Thanks guys.
I'm very intrested in supermicro, but it's hard to find in my country(asus- easy) and i have no expirience with xeon







so asus i think be good choice for beginer (but xda9 maybe be next







).
btw. i have one more question about asus. for new 22nm xeon i need burn new bios in MB. but as i know for this you need old xeon. but i don't have it (and can't take it). can i use for this operation desctop 2011 cpu (for exp. 3930K)?


----------



## squidman

Others know better, but you might be able to put (1!) only in cpu slot 0 (main slot) (that is iff you have one lying around). For 2 cpu's of course you need the xeon processors. Or you could just order a cheap xeon off of ebay (or 2! cost less than 1 i7)....of course that's for the gamblers....


----------



## squidman

OK need help with premiere pro cs6 on this POS!

Have given premiere a separate scratch disk (hybrid ssd/wd Green), and trying to render a simple h.264 project (around 45 minutes), and will have to fix audio afterwards. But damn, upon rendering, whether with premiere or export to media encoder, the 16 cores/32 threads just run at 17-24%!! ***!!! it's GLACIALLY slow (no filters, etc applied..straight up encoding!). So what the fu"#¤%%#¤%/¤#!!! How can it be SLOWER than my 4 yr old lappie when I was doing this with Vegas and smart-rendering? At this point seriously considering reverting to older versions of Vegas and smart-rendering, or (God forbid!), power director on the lappie with the intel acceleration. At the moment this simple render will take over 1.5 hrs! *** "#¤%¤!! For a 45 minute vid! Premiere is running off of an ssd, the scratch disk is off of a hybrid ssd/hdd combo, and I have 32 gb ram most of which is unused! ***! "¤#%/¤#%"#¤/&&!!!!









Any ideas? In fact the output render is going to a 3rd ssd, not that it's helping...again, agonzingly slow.....

My bad, a bit off topic, was hoping blackened tush or other nle/vid experts could chime in here....

Edit:
running Vegas 12.xxx, (no smart-render), getting about the same render times....sigh! So around 1.5x render x actual times....I was hoping we were getting faster than that by now.....Vegas using around 60-80% of cores....


----------



## Element2k3

Got a question for the z9pe d8 ws community: is it better to get let's say 4x8gb ECC or 8x4gb ECC? I know that 4x8 gives me the chance to double it later. But from a point of view of speed (seen that it is a quad-channel), which one is better?

@Squidman, that is easy to explain (and you really should have done some research before getting expensive hardware): Premiere isn't optimized for multi Cpu with 32 threads and low speed. Actually, it wants the opposite - less but very fast. So if you bought the cpus to speed up Premiere, it might have a big waste of money.


----------



## anubis1127

Quote:


> Originally Posted by *squidman*
> 
> OK need help with premiere pro cs6 on this POS!
> 
> Have given premiere a separate scratch disk (hybrid ssd/wd Green), and trying to render a simple h.264 project (around 45 minutes), and will have to fix audio afterwards. But damn, upon rendering, whether with premiere or export to media encoder, the 16 cores/32 threads just run at 17-24%!! ***!!! it's GLACIALLY slow (no filters, etc applied..straight up encoding!). So what the fu"#¤%%#¤%/¤#!!! How can it be SLOWER than my 4 yr old lappie when I was doing this with Vegas and smart-rendering? At this point seriously considering reverting to older versions of Vegas and smart-rendering, or (God forbid!), power director on the lappie with the intel acceleration. At the moment this simple render will take over 1.5 hrs! *** "#¤%¤!! For a 45 minute vid! Premiere is running off of an ssd, the scratch disk is off of a hybrid ssd/hdd combo, and I have 32 gb ram most of which is unused! ***! "¤#%/¤#%"#¤/&&!!!!
> 
> 
> 
> 
> 
> 
> 
> 
> 
> Any ideas? In fact the output render is going to a 3rd ssd, not that it's helping...again, agonzingly slow.....
> 
> My bad, a bit off topic, was hoping blackened tush or other nle/vid experts could chime in here....
> 
> Edit:
> running Vegas 12.xxx, (no smart-render), getting about the same render times....sigh! So around 1.5x render x actual times....I was hoping we were getting faster than that by now.....Vegas using around 60-80% of cores....


Sorry, I've never used premiere, it sounds like the application is not very well optimized for multi-thread.

Quote:


> Originally Posted by *Element2k3*
> 
> Got a question for the z9pe d8 ws community: is it better to get let's say 4x8gb ECC or 8x4gb ECC? I know that 4x8 gives me the chance to double it later. But from a point of view of speed (seen that it is a quad-channel), which one is better?


I would vote 8x4GB for the reason you mentioned, quad-channel bandwidth. As long as you only need 32GB RAM, that is the route I would go.


----------



## Element2k3

Thanks! Any idea/experience what the speed difference would be?

Up till now, I used to render and run simulations with 16gb. And with good management I rarely ran out of memory - but obviously now that I'm jumping on multi-cpu, I don't want to compromise. And 8x8gb ECC easily cost the double.


----------



## BlackenedTush

Quote:


> Originally Posted by *squidman*
> 
> OK need help with premiere pro cs6 on this POS!
> 
> Have given premiere a separate scratch disk (hybrid ssd/wd Green), and trying to render a simple h.264 project (around 45 minutes), and will have to fix audio afterwards. But damn, upon rendering, whether with premiere or export to media encoder, the 16 cores/32 threads just run at 17-24%!! ***!!! it's GLACIALLY slow (no filters, etc applied..straight up encoding!). So what the fu"#¤%%#¤%/¤#!!! How can it be SLOWER than my 4 yr old lappie when I was doing this with Vegas and smart-rendering? At this point seriously considering reverting to older versions of Vegas and smart-rendering, or (God forbid!), power director on the lappie with the intel acceleration. At the moment this simple render will take over 1.5 hrs! *** "#¤%¤!! For a 45 minute vid! Premiere is running off of an ssd, the scratch disk is off of a hybrid ssd/hdd combo, and I have 32 gb ram most of which is unused! ***! "¤#%/¤#%"#¤/&&!!!!
> 
> 
> 
> 
> 
> 
> 
> 
> 
> Any ideas? In fact the output render is going to a 3rd ssd, not that it's helping...again, agonzingly slow.....
> 
> My bad, a bit off topic, was hoping blackened tush or other nle/vid experts could chime in here....
> 
> Edit:
> running Vegas 12.xxx, (no smart-render), getting about the same render times....sigh! So around 1.5x render x actual times....I was hoping we were getting faster than that by now.....Vegas using around 60-80% of cores....


Squidman,

I don't agree with Elernment 2K3. Adobe advertised advanced 64bit hyperthreading from v CS5.5 and it also touted the _Mercury Playback Engine_ with its additional GPU acceleration that takes advantage of CUDA, Open GL & Open CL.

Now you have an NVIDIA card...but I haven't heard you mention enabling the Mercury Playback Engine for your project. Have you done that? (_Project>Project settings>General>Video rendering and Playback>Mercury Playback engine GPU Acceleration_). You might have to manually get Premire to sniff out your GPU first.

If you're doing Red Digital Cinema high def vid then a RAID0 target drive is crucial (SSDs are known for read not write throughput). AVCHD wouldn't need the RAID target; just faster processors. I target my RAID10 array and use SSDs for anything where reading is a priority - like project source files, system/application launch, and for caching.

BT


----------



## BlackenedTush

Quote:


> Originally Posted by *anubis1127*
> 
> I emailed them this morning, we'll see. I can just deal with it, but it's annoying. I wouldn't care if it were a $150 z77 board, but it was a bit more than that.


What a bummer. I feel for ya. The most amazing yet problematic board I've ever seen. The over/under achiever.

BT


----------



## Element2k3

Well, you can agree with me or not. Just check http://ppbm5.com/index.html for real-world benchmarks and forums like cgtalk and creativecow.


----------



## ignoramus

Ok, i cant find the information anywhere on how to make start-ups smoother.

From a cold start it powers up then powers down and back up again and then attempts to start 4 or 5 times (jumping back and forth between the initial ASUS screen and intel boot somethingorother) before it loads windows and crunches more data than chuck norris ever could.

Did anyone find a solution? Are there devices/options that i can disable in the BIOS to make this go away? tbh its not that big a deal - i dont plan on restarting much more than once a month and the startup time is still under 2 minutes but i would just prefer it was sorted, ya'know?

NB. Final cinebench score is 18.8. Thats with 2 old 7200rpm laptop HDDs in raid 0 and memory @ 1333. My mem doesnt boot at 1600..

Also make sure your intel chipset drivers are up to date. i dont know if it was raiding the HDDs which gave me a 0.5 increase in cinebench but it may have been better chipset drivers. Note: intel's auto update feature on their website never works for me but driver genius always finds the latest ICDs.

Edit- is a [email protected] score of 70K about right? i have 2x 2665s and a GTX 460.


----------



## TheBlademaster01

Yup, 19.2-18.6 in cinebench is what I score.


----------



## BlackenedTush

Quote:


> Originally Posted by *Element2k3*
> 
> Well, you can agree with me or not. Just check http://ppbm5.com/index.html for real-world benchmarks and forums like cgtalk and creativecow.


Hey Element 2K3,

What we can expect performance wise has a lot to do with which version of Adobe stuff we got and which program we're talking about. They all have their own programming groups within Adobe; but there's more sharing going on between them. I know that Adobe is continually trying to improve performance and has made great strides in Premiere AE and Photoshop... especially since CS5.5. And CSCC is constantly changing with each minor update. The conclusions you brought up are subjective and based on CS5

I know Creative Cow, CGTalk, Tom's Hardware, et al have peeps who relay their own experiences and apply benchmarks etc.. But I've found that it's really difficult to apply whatever lessons they may offer as definitive principles for all situations. My own results have varied and the reasons why aren't usually clear.

Telling squidman that he should have done his research isn't particularly helpful. Firstly because we have no way of knowing that he did or didn't do research; but it would seem obvious that if he is building a workstation for video, that he would have at least done some. And secondly, because the software is constantly changing and the best hardware to get for it is a moving target. Everybody has their own strategies for dealing with this phenomenon. Personally, I didn't aim for optimizing my machine for CS4/CS5, which were the prominent versions when I started... I aimed a little ahead for where I heard that the future versions were going (CS5.5 & CS6).

So it's better to take it all in... take what Adobe says, take what users say, apply the best practices, and then see where you end up.


----------



## squidman

Element2k3 or whatever it was...forgot the quotes...

Yeah you may be right there! But I do a bit of 3d rendering (vray, etc) and then I REALLY needed those extra spicy cores! Actually, it's been a while since I did a longer render on the laptop, so not sure iff 1 to 1 (got a bit close to that ratio finally) is actually that bad! Still a bit disappointed no NLE has really taken advantage of the intel fast render thing, (tried it out on premiere? a while ago...the sdk plugin) but just got very strange colors and crashes.

As far as I know all 16 cores are actually running at 2.3 ghz, and things like Vue, Vray, etc. are just so much faster it's not funny. So I can live with premiere rendering a bit slower (still almost 1:1! So not too shabby...but was hoping we were even faster with our asus mobos and 16-24 cores!)

Intel quicksync never really worked for me...tried the SDK etc...was all wrong. Shame as it's really blindingly quick and with h.265 and 4k we may need that. No I don't do only video, a bit of everything (generalist freelance, 3d, compositing, etc etc).


----------



## squidman

Yes Mercury is enabled...had to do the hack on my laptop with it's puny 2gb geforce gt540m (and similar hack for after effects cuda thingy...whatever it was...raytracing? forget)...but doesn't seem necessary with master collection CS6 (and gtx 770). I'll cut off my left cojone before I go to adobe cc!

Not quite optimized here (frankly still learning the Adobe suite...pretty experienced with PS, Illustrator, AE, Flash (hah hah..obsolete!) but just learning the ropes with Premiere..only just getting my raid setup in order (4x4tb in raid 1+0) and struggling with various Corsair problems (weeelll, iff they want to call themselves Corsair, no quarter given! Been pretty vocal and harsh on the forums, not that that will help me....sigh....)

Thanks for the feedback guys (and gals? what no hot chickas on overclock.net?)...keeping it surreal...


----------



## squidman

Quote:


> Originally Posted by *BlackenedTush*
> 
> Squidman,
> 
> I don't agree with Elernment 2K3. Adobe advertised advanced 64bit hyperthreading from v CS5.5 and it also touted the _Mercury Playback Engine_ with its additional GPU acceleration that takes advantage of CUDA, Open GL & Open CL.
> 
> Now you have an NVIDIA card...but I haven't heard you mention enabling the Mercury Playback Engine for your project. Have you done that? (_Project>Project settings>General>Video rendering and Playback>Mercury Playback engine GPU Acceleration_). You might have to manually get Premire to sniff out your GPU first.
> 
> If you're doing Red Digital Cinema high def vid then a RAID0 target drive is crucial (SSDs are known for read not write throughput). AVCHD wouldn't need the RAID target; just faster processors. I target my RAID10 array and use SSDs for anything where reading is a priority - like project source files, system/application launch, and for caching.
> 
> BT


No svartaraeva, still on 1080p here...via my gopro2 and original galaxy note....waiting for the blackmagic 4k camera to come out (although that poses some storage problems! actually even the pocket cam does...). I'm still pretty much in the 'home/vid for family' camp, although i was an early adopter of hdv (canon hv20 and hv30) and follow all the websites pretty much daily. Still waiting for compressed RAW in a viable form and for storage to come down in price. Installing the 4x4tb WD Red in raid 1+0 tonight, but already stressed that that isn't nearly enough capacity. And no idea how to back it up (have two older NAS, but in the same room, not a proper backup). Iff I had the ball$ and the dough for storage, I would just start shooting straight away on a blackmagic m43, but the cost of shooting RAW in terms of SSD's and storage is just prohibitive. Need to wait for some form of compressed RAW. But really would like to see 2.5k or similar in future cameras to be able to stablize handheld shots! Have used a bollywood steadycam, and frankly, it helps, but a lot of extra weight and takes a lot of practice (and effort!).

Sorry iff you thought I was a pro! But am still at amateur level here..mostly in publishing, not cinema....can do a nice book layout for you tho' if you want!

PS some will ask..why does squidman need 3d programs and compositing etc for publishing? Well it's because I suck at illustrating...I can't draw/illustrate to save my life! But with these programs I can CHEAT! And cheating is good:thumb:....especially if you need a 3d composite from plates for a bookcover! A bit of photoshopping, a bit of AE, etc....voila! As long as the printer is happy (don't get me started on editors and printers!)


----------



## squidman

Quote:


> Originally Posted by *ignoramus*
> 
> Ok, i cant find the information anywhere on how to make start-ups smoother.
> 
> From a cold start it powers up then powers down and back up again and then attempts to start 4 or 5 times (jumping back and forth between the initial ASUS screen and intel boot somethingorother) before it loads windows and crunches more data than chuck norris ever could.
> 
> Did anyone find a solution? Are there devices/options that i can disable in the BIOS to make this go away? tbh its not that big a deal - i dont plan on restarting much more than once a month and the startup time is still under 2 minutes but i would just prefer it was sorted, ya'know?
> 
> NB. Final cinebench score is 18.8. Thats with 2 old 7200rpm laptop HDDs in raid 0 and memory @ 1333. My mem doesnt boot at 1600..
> 
> Also make sure your intel chipset drivers are up to date. i dont know if it was raiding the HDDs which gave me a 0.5 increase in cinebench but it may have been better chipset drivers. Note: intel's auto update feature on their website never works for me but driver genius always finds the latest ICDs.
> 
> Edit- is a [email protected] score of 70K about right? i have 2x 2665s and a GTX 460.


Are you on 'spicy chips'/Engineering sample chips' or normal retail ones? My ASUS z9pe-d8WS still restarts several times (am on spicy chips - these days it only restars 2-3 times), but usually starts. My cinebench was around the same for e5-2665's. Never was able to install raid drivers once I started installing win7 ultimate 64, was a bit of a nightmare. Bit of a late night here per usual (always when I go on this site! One good bottle of vino rosso down!), but I recall trying to install drivers under the win7 installation was an absolute nightmare using the ad-hoc method (installing drivers during OS installation). Never was able to get the intel IRSTe to work, but finally got the Marvell chipset to work AFTER OS installation. Ditto LSI chipset, never worked.

Bit zonked here, but you've absolutely found the right place for help (some details on your board, chips, etc would help!), have nothing but props for the guys on this forum! Good luck!


----------



## x86box

I have the Z9PE and x2 2.7ghz 2680. w/ 64GB Ram. I'm working with x1 Titan GPU. Atto 6F0, 16 channel internal Card w/ x12 3TB in RAID 0. 1800MB/s. Myricom 10Ge, BlackMagic 4k, or decklink Duo.

Liquid cooled with x2 CPU / x1 Titan / x2 chips on Atto Raid card. Rackmount in SuperMicro case, Swiftech MCR220 Drive. 52ºC full load, low fans. GPU 45ºC full GPU load. Cinebench is around 20.

I can say performance for me is very good in AE, Davinci Resolve, Colorfront, etc.. I'm seeing that Red Raw footage has absolutely no leverage on GPU, only the Red Rocket card, otherwise CPU only. It's one of the few things that peg my CPU at 100%. I'm decoding Full res at 22fps. It's good to know because I'm only getting 24fps with my Red Rocket card installed. Real-time Playback on 1/2 Res, w/o Rocket card. I barely ever use the Red Epic though, 1-2 times per year. Same story with 1 decode of GoPro Hero 3. Alternatively, I'm using the Sony F65 daily, which decodes at 28fps out of Davinci, and is GPU based only, ColorFront Decodes 29fps with Titan also. Full Res playback in all applications.
This board allows me to run a vast amount of programs at the same time, which for me is very High Performance. I've had two VM's going, FCP. Davinci and Colorfront, as well as 1/2 dozen other programs. x12 instances of MpegStreamclip going from Go Pro Hero 3 to ProRes does the trick and pegs 100%. Purely, the single most critical part of my build is the x12 Drive Raid 0. And the CPU's trailing at a close 2nd. The RAID creates so much flexibility especially in a Post environment. This the top build of this board I have, also the only liquid one. I have a clone of it thats aircooled and another one with 2630's. Very pleased with this board.

BTW, I do not run windows. OSX only. I hope that 10.9 improves performance of my Titan.


----------



## squidman

So how do you ingest/process Gopro footage (assume that's captured in protune mode)? Have you actually found a LUT online that you can share with the rest of us poor amateurs so that we don't need to go through the painful ingest to cineform->expand to stupidly large file size/color correct->reincode to h.264? Would love to hear! That is a really really annoying and moronic process!

Nice rig(s)! ..cough cough (show-off!)...









Looking for my arri alexa and phantom flex, they're around here somewhere amongst the debris of the flat!

Hey mate, why on earth don't you run Assimilate Scratch while you're at it?

Seriously tho', share with the community (as you're a pro in DaVinci etc) how to ingest Gopro protune footage, color correct etc without us amatuers going the gopro route. You could make a top blog on that, know I would follow.


----------



## x86box

I ingest with a usb3 reader, If I'm making editorial files that are Avid dnx or Pro Res, and log is preferred. I use the Mpeg Streamclip to go to ProRes, and make multiple instances of it, usually 6-12. Its a manual process depending on how preoccupied I am with other things. But i can process them simultaneously, which speeds it up tremendously. For DNX or If i need to use a grade(LUT) I use Davinci. I'm a Colorist, and I prefer grading from log instead of starting with Log --> Linear conversion.

On Windows, if staying Log FFmpeg, should offer the same results with multiple instances. I'm not a windows user at all.


----------



## squidman

Thanks! But that is for gopro protune footage? I've messed around with Avid but gave up (non-intuitive GUI)..but the point I was getting at, the cineform presets for Gopro/workflow using the gopro footage require that you ingest, convert the h.264 (to cineform) , then use a preset on the gopro software (the LUT I'm after), (these are all now in gigantic friggin' cineform files), then recompress back to h.264. So what I'm after of course is a LUT or basic color correction methodology (**** man, whether it's in Premiere, Speedgrade, Vegas, or Davinci), to avoid the gigantic files in the intermediate conversion to cineform. As an old school photographer (using 120 film and grey cards!) I guess I could actually go back to using grey cards, but it's a bit of a hassle and the old grey cards aren't always that reliable. Depends on the angle etc.

So as a colorist (no doubt also on a 10-bit 4k display!) what can we as amateurs do to go from the protune mode in the gopro2/gopro3 to recreate (sort of) what we see and the eventual output (assume that would be sRGB on the web for most viewers). Have had this prob with the early hdv camcorders, no matter how we calibrate monitors when editing, there is no color info for the cameras, ditto output (equivalent to color spaces we use in printing/publishing)...**** I guess there's REC 709, but nobody actually tests and publishes the actual color spaces of the cameras.

See my problem? Not after a super stylistic grade, just want (as close as possible!) what I see to be conveyed to the viewer. So was hoping you with gopro3 experience maybe had a lut or workflow you could share with the community.

Yes I have a ****ty older monitor, but yes i calibrate it regularly with a ****ty colorimeter (i1 display pro), and yes their software is pretty frigging broken! Have no idea of the deltas, but I figure it's good enough for amateur stuff. Would pick up a better display, but no point, waiting on 4k ones to come down in price over the next 2-3 years.

Still coming to grips with LUTs etc, am most used to ICC profiles of course here! But damn they could have come up with some standard color spaces for camera->screen->final output a long time ago! Remember whinging about that 5 yrs ago on dvxuser etc...


----------



## x86box

The simple truth is that every monitor on the entire planet is different. And when you are processing footage there is no simple Lut. If you calibrate your monitor to rec709, then the monitors that are 709 will view your work will look similar. Which is primarily Televisions. Computer Displays are a different story, and are almost always never set for REC709.


----------



## BlackenedTush

Quote:


> Originally Posted by *x86box*
> 
> This board allows me to run a vast amount of programs at the same time, which for me is very High Performance. I've had two VM's going, FCP. Davinci and Colorfront, as well as 1/2 dozen other programs. x12 instances of MpegStreamclip going from Go Pro Hero 3 to ProRes does the trick and pegs 100%.


x86box
Wow I'm impressed with what you've done with this board! Yeah. the multitasking flexibility is probably where I find my greatest benefit. How do the virtual machines fit in?
Quote:


> Originally Posted by *x86box*
> 
> Purely, the single most critical part of my build is the x12 Drive Raid 0. And the CPU's trailing at a close 2nd. The RAID creates so much flexibility especially in a Post environment.


12 drives in RAID0 zero is pretty sick! Amazing actually ... I see that your controller has a claimed throughput of 9.6GB/sec. How did you arrive at 12 being the number, rather than maxing it at 16? Physical room or cost/benefit decision? Are you using SAS drives? And have you experimented with stripe/block size at all?
Quote:


> Originally Posted by *x86box*
> 
> BTW, I do not run windows. OSX only. I hope that 10.9 improves performance of my Titan.


So it sounds like you _hackintoshed_ this board. I am doubly impressed. I had entertained the idea myself before building, but then decided against it. Any problems with that?

Good to have you in the forum...
BT


----------



## BlackenedTush

Quote:


> Originally Posted by *squidman*
> 
> Sorry iff you thought I was a pro! But am still at amateur level here..mostly in publishing, not cinema....can do a nice book layout for you tho' if you want!
> 
> PS some will ask..why does squidman need 3d programs and compositing etc for publishing? Well it's because I suck at illustrating...I can't draw/illustrate to save my life! But with these programs I can CHEAT! And cheating is good:thumb:....especially if you need a 3d composite from plates for a bookcover! A bit of photoshopping, a bit of AE, etc....voila! As long as the printer is happy (don't get me started on editors and printers!)


Ya fooled me squidman Haha..
Yeah I hear ya about 3D and Illustrator. Actually, I can publish books too, so i wouldn't need one from you thank you very much. Just graduated from the Art Institute last Thursday with a degree in EFX & Motion GFX. As a requirement for graduation I had to publish a 60-page process book, website, DVD, bizz cards, artsy resume, and a 60" x 40" display backdrop for a portfolio show...

I'm a pro in illustration and graphic design; and I'd _like to call myself a pro_ in VFX and Motion GFX... Truth is, that will come with experience & people being willing to trade money for my services. This compositing thing and 3D modeling are relatively new for me... and I'm not a video expert by any stretch of the imagination; but being able to illustrate really comes in handy for matte painting and set extensions. And I love how camera mapping of matte paintings onto 3D models really brings them alive.

But to keep it on topic, I think this Z9PE-D8 machine had just as much to do with me graduating. Haha

BT


----------



## x86box

I had to go with x12 because it fits in a Supermicro Rackmount Case with the Radiator just behind them. I couldnt fit x16 in there with the Aquacomputer Controller without putting them elsewhere in the case. x12 is good enough. I was only getting around 1400MB/s with 2TB 7200rpm 64MB cache. Now I'm using 3TB, I get around 1800MB/s when its empty. The Atto isnt the fastest controller, I want to swap it for a LSI to see if I can get better performance at Raid 5. I get about 250MB/s with Raid 5/6. Thats not enough for F65 4k full res playback. I need at least 1GB/s for that while coloring and transferring and rendering. I use the 4th port for a external SAS connection to other RAID chassis.

I use the VM for various things, testing 10.9 Maverick, running debian for networking security tools, and Windows 7 for Camera control and Aqua computer interface software. Nothing really intense. VMware doesnt support PCi-e so its very limiting for most of my applications.

On the front I have a 2.5" Dual hotswap bay for a collection of SSD's that have full blown Win7, debian, 10.9 and backup copies of 10.8.4. I actually would say that OSX runs better than windows does on this board. At least early on, for me. But I've been running os x since 10.4 on PC hardware, its easy for me to diagnose.


----------



## beaker7

Just curious why you guys are all using water to cool non (or slightly) overclocked processors?

My E5-2687w's running balls out on air are like mid 60's C.


----------



## x86box

Because GTX680 or Titan run at 85C under full GPU load and Mid 60s on CPU with Air. My system runs at 45-50C full load Liquid Cooled, 40-45C on GPU at Full load, Atto Raid card from 70C+ down to 35C. I'd say thats a significant difference. Plus I need to be silent in my line of work.


----------



## lowfat

Quote:


> Originally Posted by *beaker7*
> 
> Just curious why you guys are all using water to cool non (or slightly) overclocked processors?
> 
> My E5-2687w's running balls out on air are like mid 60's C.


I top out @ 36C and it is pretty much dead silent. I watercool everything including my server.


----------



## beaker7

Quote:


> Originally Posted by *x86box*
> 
> Because GTX680 or Titan run at 85C under full GPU load and Mid 60s on CPU with Air. My system runs at 45-50C full load Liquid Cooled, 40-45C on GPU at Full load, Atto Raid card from 70C+ down to 35C. I'd say thats a significant difference. Plus I need to be silent in my line of work.


Quote:


> Originally Posted by *lowfat*
> 
> I top out @ 36C and it is pretty much dead silent. I watercool everything including my server.


Right but even those temps are well within spec. ( ~80's GPU, 60's CPU )

I used to run water on workstations but stopped due to the hassle and Intel going with locked multi's on Xeons. Just put mine in a noise canceling rack with the rest of the kit and its dead silent







Hell of a lot easier than having to drain the loop to switch out some gear.


----------



## x86box

My system is not always in one place. My environment changes constantly, everyday in fact. On Air cooled rendering out of Davinci during summer would lock up my system when it reached 100C. My Raid card would eject the RAID if temps reached 75C. Two very bad problems.

In the same environments, My GPU would reach maximum 60C with all fans on low with Liquid cooling. My Raid card no higher than 45C. I have no choice but watercooling. Or the alternative is running max fans 'all the time', just to keep up. I can't run Aircooled because I'm on Movie Sets where we are recording sound. The loop also gets changed very rarely. All my GPU's have quick release adapters so the GPU can be released in 10 seconds or less, only losing a few milliliters each time. I haven't drained my loop since the system's Full rebuild in June, previously I didnt drain the loop since November 2012.

Water is not the most flexible option, but for me its the quietest and I've worked around all the drawbacks.


----------



## EndlessRed

Quote:


> Originally Posted by *beaker7*
> 
> Just curious why you guys are all using water to cool non (or slightly) overclocked processors?
> 
> My E5-2687w's running balls out on air are like mid 60's C.


Two less fans, quieter operation. Plus I was never a fan of mounting increasingly heavy and large heatsinks on the motherboard.


----------



## EndlessRed

So an update for my encounter with stuttering 3D graphics:

I got a brand new board from advance RMA. Installed it. Still stutters.

I hate to believe it, but I think my ES chips are lemons.


----------



## AndyE

2 questions:

1) Z9PE-D8 WS support of IB Xeons:
2 weeks ago I got 2x IB Xeons E5-2697v2 (12c, 24t, 2.7GHz, retail version).
Despite upgrading the BIOS to the latest version (5103) the board refuses to boot with the CPUs. No screen output during the boot process, diverse Q-codes are visible but the boot process stops with Q-code 00. Replacing the new Xeons with the previously installed SB Xeons E5-2687W solves the boot problem. The mb starts normally.

I had no problem whatsoever to use the new IB Xeons with the ASUS Z9PE-D16 (BIOS 5105).

Question: has anybody successfully used any retail model of the new IB Xeons with the D8 motherboard?

Second question:
We do have a discussion (and here) over at the H-forum about the performance impact of the NUMA BIOS setting. Those with a Supermicro motherboard experience better folding performance when the BIOS setting NUMA = enabled is being used. My experience with the 2 Asus boards I have (Z9PE-D8 and Z9PE-D16) is the opposite. Folding performance is better when the BIOS Setting NUMA = disabled is being used. Independent which CPU is in the system (SB Xeon or IB Xeon)

Could someone be so kind and share his folding performance with NUMA enabled/disabled ?
It doesn't need to scientifically accurate at this point. Just average the TPF times of 3-5 timesteps with either option.

Alternatively, you could also use y-cruncher, a small, highly efficient program to calculate huge numbers of PI digits. It has a benchmark mode. Just select a number of digits to calculate and run it with either NUMA setting.

Your input would be helpful to find out if Supermicro and Asus potentially implemented the NUMA BIOS setting differently.

Thank you,
Andy


----------



## arvidab

I will share my numbers with NUMA disabled/enabled. But not until Saturday probably.


----------



## ZDngrfld

I'm able to test out the NUMA settings on my Supermicro board with IPMI and my Z9. I should have some time this weekend to do it.


----------



## arvidab

Regardless if NUMA is on or off, the 8101 it's folding right now, tpf stays at 16:28-16:30.

Was wrong, it was ~16:20 with NUMA disabled.


----------



## TheBlademaster01

NUMA made a lot of difference for me on 8103.


----------



## arvidab

Quote:


> Originally Posted by *TheBlademaster01*
> 
> NUMA made a lot of difference for me on 8103.


Was it better on or off?


----------



## TheBlademaster01

Better off. Actually rendering is better with NUMA off as well (C4D that is). So UMA is better (uniform access).

What I've noticed is that an app is bandwidth sensitive or not.

If it's sensitive, NUMA will make a difference. If the program is not coded for it, then it turns out worse (messes up scheduling?). If it's coded for it, there should be a benefit.

If it's insensitive it won't really make a difference from my experience.


----------



## psyq3212

Quote:


> Originally Posted by *AndyE*
> 
> 2 questions:
> 
> 1) Z9PE-D8 WS support of IB Xeons:
> 2 weeks ago I got 2x IB Xeons E5-2697v2 (12c, 24t, 2.7GHz, retail version).
> Despite upgrading the BIOS to the latest version (5103) the board refuses to boot with the CPUs. No screen output during the boot process, diverse Q-codes are visible but the boot process stops with Q-code 00. Replacing the new Xeons with the previously installed SB Xeons E5-2687W solves the boot problem. The mb starts normally.
> 
> I had no problem whatsoever to use the new IB Xeons with the ASUS Z9PE-D16 (BIOS 5105).


I suggest you contact ASUS directly (here in Germany I get some kind of "premium" support for the Z boards) - I have no C1 IvyTowns right now to test (this is the retail stepping) but it is very weird that there is a difference, since D16 5105 and D8 5103 should share the same QPI and MRC as well as microcode update 415 which should handle retail IvyTowns.

If anything, ASUS probably has BIOS with the MRC 2.00.xxx which is already out (but was obviously late to make it to 5103 BIOS)


----------



## AndyE

Thanks.

I am already in contact with Asus Support.
They negate the problem currently - according to their info it should work in the D8 board

They asked for the sSpec of my CPUs which I sent.

BTW, based on PNs I got, it looks like I am not the only one having issues with this combination D8 / retail IB Xeon.

Will you have any retail IB CPUs soon you could test your D8 board with?

BTW. the Asus M/B D16 reports for the E5-2697v2
(the D8 board doesn't start so there is no reading from this board)
CPUSig: 306e4
Microcode: 415

rgds,
Andy


----------



## AndyE

Update:
Asus Support just sent me a mail, that they could start the board with 1 CPU, 1 DIMM and the GTX Titan.
No info yet on dual CPU boot.
Strange, as my system did not start with a single CPU, single DIMM and the Titan (plus some other graphic cards)

Andy


----------



## psyq3212

I might have a retail one in the next few weeks so I will let you know.

You might want to test the board with another brand of ECC DDR3. I think the only big difference between D8 and D16 is that D8 has "overclocking" RAM settings, which imply that they had to "tune" the MRC code for it. It could be that they just screwed something up which could manifest as incompatibility between CPU and the RAM.

Also, another thing to try is to plug the D16 BMC in D8 (you will need to re-flash its firmware with D8 image, otherwise it will not work just as a simple swap) and remotely connect and check if there is anything logged in the error log (it might not be CPU itself, but memory training or something entirely different).


----------



## TheBlademaster01

psyq, AndyE

Post a nice score in this thread pls?

http://www.overclock.net/t/1430288/maxon-cinebench-r15/










Also, if those chips ever need a new home you know whom to call right







?


----------



## poulk

Quote:


> Originally Posted by *AndyE*
> 
> 2 questions:
> 
> 1) Z9PE-D8 WS support of IB Xeons:
> 2 weeks ago I got 2x IB Xeons E5-2697v2 (12c, 24t, 2.7GHz, retail version).
> Despite upgrading the BIOS to the latest version (5103) the board refuses to boot with the CPUs. No screen output during the boot process, diverse Q-codes are visible but the boot process stops with Q-code 00. Replacing the new Xeons with the previously installed SB Xeons E5-2687W solves the boot problem. The mb starts normally.
> 
> I had no problem whatsoever to use the new IB Xeons with the ASUS Z9PE-D16 (BIOS 5105).
> 
> Question: has anybody successfully used any retail model of the new IB Xeons with the D8 motherboard?


damn i'm only decide to buy asus D8 2x 12 core xeon (ES) and already ordered waterblock for this mb. and such bad news. anyone try use new 12C xeons with this mb already? or i can have the same problem too?


----------



## psyq3212

AndyE is using retail IVB-EP Xeons, so this might be the difference.

ES and QS IVB-EP Xeons (B1 and C0 stepping) so far were found to work.

Retail stepping is C1 (which was also used in limited quantities in QS phase) so the difference might be stepping-related.

In any case, if confirmed and not just board-specific the problem with retail Xeons is momentary - I am sure ASUS will issue an update since there is no way they are going to skip support for the entire family of CPUs.


----------



## psyq3212

Quote:


> Originally Posted by *TheBlademaster01*
> 
> psyq, AndyE
> 
> Post a nice score in this thread pls?
> 
> http://www.overclock.net/t/1430288/maxon-cinebench-r15/


Just posted my results - I am getting 3098 cb, which I suppose is what is expected from the 2697 v2








Quote:


> Also, if those chips ever need a new home you know whom to call right
> 
> 
> 
> 
> 
> 
> 
> ?


I will let you know, as soon as I get hold of Haswell EP Xeon ES2 samples


----------



## arvidab

Quote:


> Originally Posted by *psyq3212*
> 
> AndyE is using retail IVB-EP Xeons, so this might be the difference.
> 
> ES and QS IVB-EP Xeons (B1 and C0 stepping) so far were found to work.
> 
> Retail stepping is C1 (which was also used in limited quantities in QS phase) so the difference might be stepping-related.
> 
> In any case, if confirmed and not just board-specific the problem with retail Xeons is momentary - I am sure ASUS will issue an update since there is no way they are going to skip support for the entire family of CPUs.


Which BIOS version did you use on non-retail? Which stepping were they?

Quote:


> Originally Posted by *psyq3212*
> 
> Quote:
> 
> 
> 
> Originally Posted by *TheBlademaster01*
> 
> psyq, AndyE
> 
> Post a nice score in this thread pls?
> 
> http://www.overclock.net/t/1430288/maxon-cinebench-r15/
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> Just posted my results - I am getting 3098 cb, which I suppose is what is expected from the 2697 v2
> 
> 
> 
> 
> 
> 
> 
> 
> Quote:
> 
> 
> 
> Also, if those chips ever need a new home you know whom to call right
> 
> 
> 
> 
> 
> 
> 
> ?
> 
> Click to expand...
> 
> I will let you know, as soon as I get hold of Haswell EP Xeon ES2 samples
Click to expand...









to both replies.


----------



## psyq3212

I am using C0 stepping QS, BIOS version 5103

All 5xxx BIOS-es for Z9PE D8 WS contain IvyTown support - but you will need 5103 for the QS (C0/C1) steppings.

5103 BIOS also contains microcode update 415, which is a production-grade microcode for IvyTown EP variants.

Another way to check for the stepping is CPUID family/model designation:

306E2 --> This is ES2 (B1/B2)
306E3 --> Normally you would not see these unless you have 4P models ES2 which was of B3 stepping
306E4 --> This covers QS (C0/C1) and retail (C1) steppings of the 2P EP models
306E7 --> This is D1 QS of IvyTown EX, but these CPUs are not compatible with LGA 2011 boards such as Z9PE D8 WS


----------



## arvidab

Thanks for your reply, psyq, much appreciated.


----------



## AndyE

psyq,
thanks for your list.

Hmm,
my CPU do have microcode 415 and CPUID 306E4 - So I should be in good shape. But I am not.








Ok, need to spend more time with the system next week when I am back.

rgds,
Andy


----------



## mateuszgpucomp

Hi
I use Z9PE-D8 WS with 2 Xeon E52696 v2.
All work greats, Bios 5103, memory if someone wants to use 2 CPU *MUST BE ECC*.
Non ECC memory work with one CPU (CPU must be in slot CPU1)
more test of this Xeon


----------



## AndyE

Quote:


> Originally Posted by *mateuszgpucomp*
> 
> Hi
> I use Z9PE-D8 WS with 2 Xeon E52696 v2.
> All work greats, Bios 5103, memory if someone wants to use 2 CPU *MUST BE ECC*.
> Non ECC memory work with one CPU (CPU must be in slot CPU1)
> more test of this Xeon


Thanks,
this explains the non-boot with my Corsair Dominator Platinum which are rated at 2133 (intended to exploit the 1866 MHz memory interface of the IB Xeons). If OC to 4 or 6%, the memory interface would have been running at 1977 MHz. Independent of my boot isues this road to speed seems to be blocked ...

Does anybody know where there are 1866 MHz ECC RAMs available? I haven't found one yet









Update from the Asus technican:
They sent me a RMA and I shall exchange my mobo.

rgds,
Andy


----------



## arvidab

Quote:


> Originally Posted by *mateuszgpucomp*
> 
> Hi
> I use Z9PE-D8 WS with 2 Xeon E52696 v2.
> All work greats, Bios 5103, memory if someone wants to use 2 CPU *MUST BE ECC*.
> Non ECC memory work with one CPU (CPU must be in slot CPU1)
> more test of this Xeon


Interesting. Thanks for the heads up.

@psyq3212: Are you also using ECC, psyq?


----------



## mateuszgpucomp

@AndYE:
DDR3 ECC Samsung 8GB MODEL M391B1G73BH0-CMA is the good choice, this is 1866MHz memory


----------



## psyq3212

Quote:


> Originally Posted by *arvidab*
> 
> Interesting. Thanks for the heads up.
> 
> @psyq3212: Are you also using ECC, psyq?


Yep, I am using eight Samsung M393B2G70BH0-YK0, which is 1600 MHz 16 GB DDR3 ECC stick running @1.35v

These modules have absolutely no problems running on 2133 MHz with just bumping the voltage.

Unfortunately, with my 2697 v2 Xeons I cannot get RAM to work above 1866 MHz at all, so I am using it @1866 MHz and 1.5v, which runs perfectly fine without even changing the timings.

@AndyE, you can get the same RAM in 1866 MHz variant, it is then M393B2G70BH0-CMA

It used to be several times more expensive but it looks like Samsung finally ramped up the production:

Germany:

- 16 GB ECC DDR3 1600 MHz is 130 EUR (http://geizhals.at/eu/samsung-dimm-16gb-m393b2g70bh0-yk0-a874445.html)

- 16 GB ECC DDR3 1866 MHz is 172 EUR (http://geizhals.at/eu/samsung-dimm-16gb-m393b2g70bh0-cma-a984667.html)

Basically, it is the same RAM, just rated differently. Nowdays I would just buy the 1866 MHz one, but last year this RAM was ~900 EUR per stick (!)

@edit - just noticed that mateuszgpucomp posted the same RAM info before my post







I recommend the same RAM modules from Samsung, they are great.


----------



## arvidab

Thanks for your reply, again.

Any recommendation for a fella who doesn't need 128GB, tbh 32GB is more than plenty for me, I don't fancy paying 1000+ EUR for the memory, lol...
Maybe I might as well grab some 1333 and wait until my wallet is fat enough.


----------



## anubis1127

Oh you need 128GB arvidab.


----------



## arvidab

Oh, ok.


----------



## squidman

Sorry guys, this is prbly a bit off-topic to recent threads on IB v2 processors...but I am following! At some point? the ES versions will become available, and I'll have to have a punt (more threads? yes please!)

Reply to blackened tush from x posts previous...

Read you there svarta ræva! This is my 3rd career change (from executive chef to finance to freelance in publishing!). So I don't have the advantage like the younger peps of having gone to full-sail or gnomon etc (8 years studying finance and in ivy down the toilet! but much happier doing this!). The really big step for me is to master nuke and more advanced 3ds (mostly because I love the plugins - fumefx, rayfire etc!) and maya workflows (and zbrush!). My impression is that most who do vfx are into zbrush, maya, and nuke, and those are sort of the industry standards (ok we should give lightwave, houdini some love, and for broadcast, c4d!). Still to be competent, pretty frigg.. intimidating learning curve...almost worse than my derivatives maths!....not to mention, Assimilate Scratch, Resolve, etc etc!

cookie out









Update: for those looking to install hdd's in the last four SAS/SATA ports, installed 4 WD Red 4GB drives in raid 1+0/10, and after much mashing of friggin keys at startup to get into the relevant ISTRe menu, finally got it working. Unlike the LSI crap for the top two sata ports, it actually works. Of course, almost 1k usd later, in raid 10 I only have 6.91 TB space out of 4x4TB drives, but have had too many bad experiences with hdd failure lately to risk raid 5. Finally got Corsair cooling link garbage to work also, drama hopefully sorted 4 weeks later! Still....stay away! Now iff I could only fit the last 5th hdd hotswap cage into my 900d somehow...without dremeling....









Goodies that are arriving soon: 2x Noctua 140mm fans NF-14 PWM, a raid card (startech card, link here..http://eu.startech.com/Cards-Adapters/HDD-Controllers/SATA-Cards/4-Port-PCI-Express-SATA-6Gbps-RAID-Controller-Card~PEXSAT34RH) for around 140 bucks, and 2x NZXT IU01 USB internal expansion pcb's...(getting the latter sent from somewhere that had it in stock was a pain the rear! Finally via ebay uk site...amazon wouldn't ship any of the above...region discriminating!). Phew...

cookie out!


----------



## squidman

Oh yeah,

I assume none of the SATA/SAS ports have port replication? Otherwise I've blown a bit of cash on a raid-card needlessly! Need to stick in 3-6 more hdd's, in jbod (have only physical space for 3 due to poor case design! don't go there!







)...

cheers


----------



## squidman

Quote:


> Originally Posted by *BlackenedTush*
> 
> Ya fooled me squidman Haha..
> Yeah I hear ya about 3D and Illustrator. Actually, I can publish books too, so i wouldn't need one from you thank you very much. Just graduated from the Art Institute last Thursday with a degree in EFX & Motion GFX. As a requirement for graduation I had to publish a 60-page process book, website, DVD, bizz cards, artsy resume, and a 60" x 40" display backdrop for a portfolio show...
> 
> I'm a pro in illustration and graphic design; and I'd _like to call myself a pro_ in VFX and Motion GFX... Truth is, that will come with experience & people being willing to trade money for my services. This compositing thing and 3D modeling are relatively new for me... and I'm not a video expert by any stretch of the imagination; but being able to illustrate really comes in handy for matte painting and set extensions. And I love how camera mapping of matte paintings onto 3D models really brings them alive.
> 
> But to keep it on topic, I think this Z9PE-D8 machine had just as much to do with me graduating. Haha
> 
> BT


Matte painting is the bomb! And thank god for Vue there (and photoshop)! not to mention the better rotoscoping in AE CS6! Still to take it to the next level, I have a feeling I'm going to have to get down and dirty and learn nuke...(sigh! much more comfortable with timeline as opposed to node based workflows! oh well....)...

Had to learn indesign on the fly (1 week!) for my first layout, but have to say it's a godsend when you get your head around it. And as per usual, printers and publisher on my ass, author and I whacked it out in 1 weekend of all-nighters...printers and publishers are pretty unforgiving! Live and learn I guess (NEVER rely on authors to get it finished in time for deadlines! Never rely on publishers to be happy!)...









Sorry for off-topic post! You guys can go back to Ivy bridge and xeon v2 now.....


----------



## psyq3212

@AndyE,

Once you get back to setting up Z9PE D8 WS with 2697 v2 (now with ECC RAM), make sure you disable Isoc in QPI configuration!

As I wrote few pages before, 12-core Ivy Bridge EP parts do >not< support isochronous traffic - and keeping this option enabled led to phantom 3-strike errors and subsequent WHEA BSOD-s. I suppose ASUS will hide this option when 12-core part(s) is detected in one of the future versions of the BIOS.


----------



## Lyuokdea

I am about to buy an ASUS Z9PE-D8 along with two Xeon 2670V2 processors (and 4 x 16 GB ECC 1600 MHz RAM).

It sounds like the system is not going to boot unless the motherboard BIOS is updated to the latest version. But I won't have any older processors to do this update with - anybody know how to ensure that the motherboard I buy now is already updated to the latest BIOS?

Also, anybody know of a memory module that is tested to work well?

Thanks,

~Lyuokdea


----------



## psyq3212

You can simply ask the seller to ensure that the board has 5103 BIOS. If it is a bigger seller they have means to flash it.

If this does not work, another alternative is to buy one of the pre-flashed BIOS chips on Ebay. This should be $10 or so in cost - and it gives you guarantee that you will be able to boot, in the worst case by swapping the BIOS chip.

Alternatively, you could also buy the SPI flash programmer for yourself assuming you have a laptop / second computer to connect the SPI flash programmer to. These devices sell for few tenths of dollars online. But this might be an overkill if you would do this only once.

Fourth option would be to hot-flash the BIOS on another ASUS board, but for this the board must have ME SPI flash protection off - ASUS switched this protection ON in their recent boards (Z87 boards have it from beginning, X79/C602 boards have this option since some months ago).

If you decide to hot-flash the procedure is:

#1 - Download the Z9PE D8 WS BIOS 5103 on the second ASUS system as well as AMI flasher for APTIO EFI BIOS-es.

#2 After you booted into Windows on the second board, very carefully remove the BIOS chip from the board. Then put the the Z9PE D8 WS BIOS chip in

#3 Flash the 5103 BIOS ROM

However, this procedure will not work if the "donor" board has ME SPI flash protection enabled and the already existing BIOS on the Z9PE D8 WS chip has the protection bit flag set. This is almost certain to happen unles the board has very old BIOS.


----------



## Lyuokdea

Thanks for the help!

Sounds like 1 and 2 are the way to go - I could eventually figure out the rest, but don't want to invest the time. How easy is it to swap out the bios chips on the motherboard? Do they simply snap off and on?

Also, is there a seller you would recommend? I assume Amazon might be able to check their stock for me, but they aren't going to flash anything, I wouldn't think.


----------



## psyq3212

In case of recent ASUS boards (I am talking about few generations at least) - the BIOS chip is placed in its socket, and can be removed by careful elevation with a flat tool, or by using a specialized chip removal tool:

Chip looks like this:



The most important part is to remove it carefully by applying equal force across the entire surface - otherwise the pins would bend, and this is to be avoided. In general, anybody should be able to remove the chip with some care and by using low force.

As for the sellers - I buy my stuff from Amazon and in most cases it comes with the latest BIOS. If it comes from an Amazon marketplace I am sure you can contact the seller as well and ask them for the version.

BIOS 5103 is already more than 6 weeks old, so I suppose big sellers with lots of sales probably stock the boards with the latest BIOS already.


----------



## lowfat

So it bios 5004 the one I want for compatibility with engineering sample SB-EPs? I forgot to flash to a different bios on my Z9PE-D8 with the 3930k I had before selling the CPU.


----------



## TheBlademaster01

Depends on which SNB-EPs. B0 won't work with BIOS 3506 and later, so no 5004. 5004 is the first to support IVB-EP though. C0/C1 SNB-EP should work with all of the BIOS versions (well, maybe not all will work with the first BIOS







)


----------



## lowfat

My IHS' are blank with no model numbers written on them at all. However they definitely are not working with 5103. Will bios 3302 for sure work with all ES SB-EPs?ttt


----------



## ignoramus

im on 3302 and can confirm they work with the early B0 stepping E5-2665s

just as long as you dont brick your board by replacing the chip and flashing up to 3302 and as long as you do actually have a SB E52600 series cpu you should be fine.

this board seems tempermental, which is why alot of people here are too scared to update the bios and are sitting in the 'if it aint broke' camp. maybe someone that has successfully flashed can suggest a best practise (minimal amount of ram, etc..)


----------



## lowfat

I had it working but didn't know this thread existed. I was having to reset the cmos all the time to get it to post. So I went and tried flashing the bios to see if it would help. Ya that didn't go well.









Well I went ahead and ordered a bios w/ 3302.


----------



## squidman

With the B0 chips, and flashing generally, it seems the intermediate flash is vital.....i.e. the one mentioned by Asus. Forget which one, have a total reinstall due to stupidly installing irste drivers...


----------



## squidman

Ok back to hdd issues....

Has ANYONE got either the LSI megaraid to work or the intel RSTe to work on the top two (sata 6gbps) slots? Spent a whole day trying raid 0 with both systems, but on windows install impossible to load the necessary c600 drivers...and I tried them all (asus both older and newer, from the intel site, etc etc). On win 7 install the system sees the virtual drive, but refuses to load the x64 drivers (32bit drivers load, but that doesn't help....just gives a bsod on restart).

Possibilites....
1) you can't raid 0 on hotswap drives (great....as far as I know, not possible to disable hotswap on the asus z9pe-d8 ws)
2) the drivers aren't signed and never will be (even intel drivers) so you need to slipstream them onto the install disc...win7 doesn't do f6 installs anymore.....
3) the drivers are missing crucial .inf or .stor files or are just broken....
etc etc

Reasons for raid 0 (2 samsung 250gb ssd's) there - backup and restore or disc imaging is going to be a pain iff I have the OS one one drive and program files on the other....restoring just the one is going to really mess with the registry.....I'm aware I won't really see speed benefits (assuming the sata III ports will be saturated).

On a positive note, the marvell controller with 2 small ssds and 2 hdds in hyperduo works a treat..even disconnected them for this last win7 reinstall, and they still showed up after reconnecting . The SATA/SCU raid (last bottom 4 sata slots) also works, had them in raid 10, but may change to raid 5 and risk it (scary thought, but 7.6GB just isn't enough).

Other...the LSI raid option seems to disable normal ahci for the remaining 4 ports (need those for jbod) so not really an option. Now iff I could only stuff my last hdd cage in my 900d case somehow (out with the dremel maybe...will have ps sticking out of case, but too bad)...will have 18 drives in there when I'm finished.

PS any harm in stacking ssd's on top of each other with velcro vis-a-vis heat? That would free up 3 hdd slots.....


----------



## sagitarius2k

Hallo All









I just puchase this board, and then i will install this in my board :

2x2690 E5
DDR 3 2133

as mention in ASUS web

8 x DIMM, Max. 64GB, DDR3 2133(O.C.)/2000(O.C.)/1866/1600/1333/1066 MHz ECC, Non-ECC, Un-buffered Memory *1
Quad Channel Memory Architecture
*For Registered Memory, Max. 256GB DDR3 1866/1600/1333/1066/800 MHz
* Refer to www.asus.com for the Memory QVL (Qualified Vendors Lists).

but the part i dont understand, why 2133 is marked as OC? so i must OC my RAM or my board to do this? and, if i buy 2133 RAM, should it work properly or it will down to 1866?

oh, which last bios i should use for this set up? thanks


----------



## Donkey1514

Anyone interested in 2x 2687w's?


----------



## TheBlademaster01

Quote:


> Originally Posted by *sagitarius2k*
> 
> Hallo All
> 
> 
> 
> 
> 
> 
> 
> 
> 
> I just puchase this board, and then i will install this in my board :
> 
> 2x2690 E5
> DDR 3 2133
> 
> as mention in ASUS web
> 
> 8 x DIMM, Max. 64GB, DDR3 2133(O.C.)/2000(O.C.)/1866/1600/1333/1066 MHz ECC, Non-ECC, Un-buffered Memory *1
> Quad Channel Memory Architecture
> *For Registered Memory, Max. 256GB DDR3 1866/1600/1333/1066/800 MHz
> * Refer to www.asus.com for the Memory QVL (Qualified Vendors Lists).
> 
> but the part i dont understand, why 2133 is marked as OC? so i must OC my RAM or my board to do this? and, if i buy 2133 RAM, should it work properly or it will down to 1866?
> 
> oh, which last bios i should use for this set up? thanks


I can't get anything over 1600MHz to post with my 2665s. I pretty much doubt that you'll get 1866MHz to work let alone 2133MHz


----------



## sagitarius2k

Quote:


> Originally Posted by *TheBlademaster01*
> 
> I can't get anything over 1600MHz to post with my 2665s. I pretty much doubt that you'll get 1866MHz to work let alone 2133MHz


hmmm, interisting, may i know which bios version you use? thanks


----------



## TheBlademaster01

Really old, 0503 or so. But I don't think that matters really looking at other users in this thread. I only know of some people here running 1866 ECC or 1866 registered. Other than that I don't know.


----------



## sagitarius2k

Quote:


> Originally Posted by *TheBlademaster01*
> 
> Really old, 0503 or so. But I don't think that matters really looking at other users in this thread. I only know of some people here running 1866 ECC or 1866 registered. Other than that I don't know.


THanks for answer. So, i assume that it is much "safer" to pick 1866 or 1600 non ecc RAM?


----------



## TheBlademaster01

Yeah, unless someone has a better suggestion here.

I use 1866 Dominator Platinums @1600 myself. It accepts XMP profile of the RAM but any other timings (even loosening them) makes the system unable to POST.


----------



## DizZz

Quote:


> Originally Posted by *Donkey1514*
> 
> Anyone interested in 2x 2687w's?


How much?


----------



## sagitarius2k

hello all, i have question regarding cooling

so, due to budget, i cant go all watercooling, so i try to pick air cooled solution as temp solution. My chose is Noctua NH-D14 se2011. The question is, will it fit with those gigantic cooler in this board? If not, can anyone suggest which air cooling can be fit and good performance too, and yes, loud is not a factor.

thanks

-edit-

nevermind, lol


----------



## TheBlademaster01

Tbh, most of these chips don't need beefy coolers. The Noctua will probably fit if you use low profile RAM (no heat spreaders).


----------



## squidman

I wonder iff it's such a great idea to have 2 giant aircoolers on the board (stressing the board). The main reason I went 2x h80is was to avoid stressing it. Unless you're folding 24/7 or rendering 24/7, is it even really necessary`? Have heard good things about the noctuas tho'...that was my alternate choice. Btw, I would stay away from the corsair watercooling kits until they sort out their firmware/software...neverending nightmare at my end......(at least for 2 coolers....)....looks pretty cool tho....









Oh yeah the h80i's are friggin loud! Even on quiet mode....ditto the corsair fans (have some noctuas coming)...


----------



## sagitarius2k

Quote:


> Originally Posted by *squidman*
> 
> I wonder iff it's such a great idea to have 2 giant aircoolers on the board (stressing the board). The main reason I went 2x h80is was to avoid stressing it. Unless you're folding 24/7 or rendering 24/7, is it even really necessary`? Have heard good things about the noctuas tho'...that was my alternate choice. Btw, I would stay away from the corsair watercooling kits until they sort out their firmware/software...neverending nightmare at my end......(at least for 2 coolers....)....looks pretty cool tho....
> 
> 
> 
> 
> 
> 
> 
> 
> 
> Oh yeah the h80i's are friggin loud! Even on quiet mode....ditto the corsair fans (have some noctuas coming)...


could you post if you already buy noctua? i wonder how big it is.


----------



## squidman

Not here yet...but those are only 140mm case fans.....the 900d case comes with 3 really noisy fans at the front...annoying as hell....







A smaller noctua cpu cooler (x2!) should be right up your alley....after all can't oc the xeons or memory (1600 mhz tops)....just check the airflow....you'll be right....







damn what a great smiley!


----------



## sagitarius2k

Quote:


> Originally Posted by *squidman*
> 
> Not here yet...but those are only 140mm case fans.....the 900d case comes with 3 really noisy fans at the front...annoying as hell....
> 
> 
> 
> 
> 
> 
> 
> A smaller noctua cpu cooler (x2!) should be right up your alley....after all can't oc the xeons or memory (1600 mhz tops)....just check the airflow....you'll be right....
> 
> 
> 
> 
> 
> 
> 
> damn what a great smiley!


as TheBlademaster01 said, memory can only run at 1600 mhz,, and you too? i mean can only run 1600 mhz? how about 1866, as this board saying without OC emblem on it, can i run it? because i still loking a "perfect" memory for this ( 1600 or 1866 )

damn, i just bought 900d case for this rig, i have no idea how load it is ( front fan )


----------



## squidman

the 900d works for me (but you need to be aware of the mobo standoffs...or you can short the board....use the hacksaw method)...and the other prob is that with a decent sized power supply, you won't fit the extra hdd on the bottom. But yeah I love the case, lots of space to work in. Not really into overclocking, but then I guess you knew that going into this with the current xeons....as far as I recall, there are 2? mobo standoffs that don¨t fit the mobo tray (which is sized for everything BUT ssi-eeb)...I tried tapping holes, but not accurate enough so had to give up on that. See my earlier threads on that.

Good luck! If you can get everything to work (wow with this mobo, pretty big iff!), it's a rocking combination...pretty much on everyone's dream-pc list...still struggling with bsod's here, etc....but when it works, it churns though vid and rendering like no tomorrow....









Case fans..weeell, not that bad, but the front fans for me are the loudest of all the components...Corsair has some quality control problems. Turns out it was the front fans and not the h80i's or rear fan that were the most annoying. Loud and loud...you can hear it when on low, so definitely not a super silent case until you replace them...but that could be mine.

Oh yeah..it's still a super sexy case! Don't regret purchasing mine! Unless you can go a mountain mods or some custom job...actually good value for money....you can see me on same username on corsair forums giving the corsair folks a lot of **** heheheheh...(deserved!)...but yes again it's the only case I would buy that's available in europe....


----------



## squidman

You're aware that this is not like slapping together a gaming case I take it? Be prepared (very prepared!) for mobo not posting, lots of frustration, everything conspiring against you (in other words, this is not a build you would use for work!)..if you don't have a lot of patience and enthusiasm for pc building, this is definitely not for you!

On the other hand...at time of writing, it is the sickest pc you can build with off the shelf parts......

Note I'm on ES chips, so it may be a lot easier build using retail chips (but if you can afford that, well you can afford a Stanford engineer to babysit your pc!! lol).

Heh heh good luck! Post some pics!

PS no xeons don't overclock (current generation)...srry! you might get a mild oc (check thread) but nothing dramatic....personally I just want it to post every time!









PPS you're also aware that this is NOT I repeat NOT a build for gaming? In fact it will pretty much suck at gaming compared with a system that costs half as much....

Now 3d rendering or video work....


----------



## squidman

Quote:


> Originally Posted by *sagitarius2k*
> 
> as TheBlademaster01 said, memory can only run at 1600 mhz,, and you too? i mean can only run 1600 mhz? how about 1866, as this board saying without OC emblem on it, can i run it? because i still loking a "perfect" memory for this ( 1600 or 1866 )
> 
> damn, i just bought 900d case for this rig, i have no idea how load it is ( front fan )


Memory...weelll just get some memory that works! Iff you need more than 64gb, well start saving up for ECC memory....it can take up to 256GB? of the expensive stuff. But only 64GB of non-ecc. I'm using the cheapest corsair vengeance I could get (4x8gb sticks - red ones) and they're working at 1600mhz (they're good for 18xx mhz but no point in pushing it). No point in going crazy as you can¨t oc anyway. One thing I would look at is getting matched memory (max your memory and budget).....i.e. kits are matched. Memory and vid cards are one thing this board appears to be sensitive to. In fact it won¨t post reliably to a pcie x16 slot with an nvidia gtx 770 (aka gtx 680)....only on the x8 slot. Recurring theme with nvidia (but again, everyone who does video/vfx/3d etc needs nvidia....e.g. mercury engine, cuda, physx, etc etc!! bummer +++)..

Tired here, but kept on writing as you are the only one apart from myself I know of that has the balls of steel to try this combination....









Edit: I repeat, you can **** up a lot of things, but the mobo stand-offs are not one of them! Otherwise a new board......and no the 900d does not fit perfectly.....you need to modify the middle stand-offs...otherwise buy a new mobo....!!!!!! Is that imprinted in your brain? Hope so....







Srry almost shorted my own board that way...


----------



## Jann

For something like E5-2660 ES chips Hyper 212 Evo is totally enough cooling. Even stress testing under Linx temps didn't go over 65-70 I think.
And while rendering, max temps are around 54.


----------



## psyq3212

By the way, Noctua is also making server coolers:

http://www.noctua.at/main.php?show=productview&products_id=67&lng=en
http://www.noctua.at/main.php?show=productview&products_id=66&lng=en

NH-U12DX i4 and NH-U9DX i4 are LGA2011 compatible and support both square ILM and narrow ILM - you get the standard Noctua benefits, but with smaller form-factor which is designed for server/workstation LGA 2011 setups.

NH-U12 DX:



The only downside is that the fan is 120mm - not 140mm like for HEDT LGA 2011 setups.

Noctua gear is not the cheapest option, but my experience with Noctua products so far has been absolutely superb.

Also, since SNB-EP and IVB-EP Xeons are not overclockable, it is hard to get to the point where these coolers would not be sufficient. This is unlike the case of HEDT CPUs such as 4960x, where with overclocking it is quite easy to blow above 200W+ TDP, and thus oversize thermal management solution is absolutely required.


----------



## squidman

Update on H80i solution...finally got it working with 2 NZXT internal usb header expanders (the h80i's need to be plugged into those, the corsair link ***** doesn't work), and now it actually boots every time and spins up without having to go to back of case and replug power to fans. Temps are around 30C on idle (21c room temperature) and pretty much stay there even rendering. Will run a pi95 stress test at some point, so despite a lot of frustration, the closed loop solution does work....but yes 2 mnths later. In fact a really good solution, controlling all 6 case fans and the h80i's, but wow that was a b!tch and a half.....

the 2x Noctua 140mm fans working ok as well...noticably quieter than the corsairs....iff I had the dough I'd switch out the rest of the 8 corsair fans for noctuas.....even the startech raid card works (marvell chip), was seen immediately by the marvell software...awesome! something actually works! Just appears as a 2nd marvell controller (assuming you're using the 4x marvell sata ports). Ditto even the SATA/SCU intel raid working in Raid 5....ok don't want to jinx it...knock on wood....

time for a homebrew....


----------



## Krobar

Quote:


> Originally Posted by *AndyE*
> 
> 2 questions:
> 
> 1) Z9PE-D8 WS support of IB Xeons:
> 2 weeks ago I got 2x IB Xeons E5-2697v2 (12c, 24t, 2.7GHz, retail version).
> Despite upgrading the BIOS to the latest version (5103) the board refuses to boot with the CPUs. No screen output during the boot process, diverse Q-codes are visible but the boot process stops with Q-code 00. Replacing the new Xeons with the previously installed SB Xeons E5-2687W solves the boot problem. The mb starts normally.
> 
> I had no problem whatsoever to use the new IB Xeons with the ASUS Z9PE-D16 (BIOS 5105).
> 
> Question: has anybody successfully used any retail model of the new IB Xeons with the D8 motherboard?
> 
> Second question:
> We do have a discussion (and here) over at the H-forum about the performance impact of the NUMA BIOS setting. Those with a Supermicro motherboard experience better folding performance when the BIOS setting NUMA = enabled is being used. My experience with the 2 Asus boards I have (Z9PE-D8 and Z9PE-D16) is the opposite. Folding performance is better when the BIOS Setting NUMA = disabled is being used. Independent which CPU is in the system (SB Xeon or IB Xeon)
> 
> Could someone be so kind and share his folding performance with NUMA enabled/disabled ?
> It doesn't need to scientifically accurate at this point. Just average the TPF times of 3-5 timesteps with either option.
> 
> Alternatively, you could also use y-cruncher, a small, highly efficient program to calculate huge numbers of PI digits. It has a benchmark mode. Just select a number of digits to calculate and run it with either NUMA setting.
> 
> Your input would be helpful to find out if Supermicro and Asus potentially implemented the NUMA BIOS setting differently.
> 
> Thank you,
> Andy


Hi Andy,

Just thought I'd chime in on this. Did some testing with this and a Supermicro LGA2011 board and findings were nearly all positive for Numa enabled. Highly optimised benchmarks like Sandra mem showed significant gains with Numa and Intel Linpack preferred Numa too. The difference was even bigger with multi-process rather than multi-thread scenarios as it seems the Win7 scheduler was quite smart with RAM allocation despite the application being un-optimised R code. Win7 has some tools to show you node NUMA memory allocation as well which can be quite interesting.


----------



## psyq3212

Microsoft optimized scheduler and memory manager for NUMA systems already with Windows Vista (in fact, even Server 2003 had some optimizations, but it was not possible API-wise to influence how the memory is allocated).

Basically what happens is this:

- Scheduler will not move the thread to another NUMA node, unless specified explicitly by the caller by changing the thread affinity

- Memory manager will attempt to allocate the memory from the same NUMA node where the thread is running, unless there is no available memory on the said NUMA node

With these two optimizations, even NUMA-unaware applications will benefit (or, better said - not be impaired). This does not work well if the application itself is changing the thread affinity without the regard for NUMA topology, in which case for such applications it might be better to disable NUMA in BIOS.

Windows 7 brought more features, namely support for processor groups (x64 versions only). With processor groups, it is possible for one process to create threads that span across the whole system if the system has more than 64 logical processors. Previously, with Vista/Server 2008 (or earlier) APIs this was not possible, because affinity APIs only had a 64-bit mask used for identifying the processor.

Processor groups will become relevant with mainstream servers in few months - because with Ivy Bridge EP 4S you can have 48 physical / 96 logical CPUs, so the application needs to use Windows 7 / Server 2008 R2 APIs in order to be able to use all of them.


----------



## Jann

Guys, how much would a Z9PE-D8 WS + 2 E5-2660 ES C0 cpus go for?
Due to doing only small personal projects at home now, I'm thinking to switching to a simple quad i7


----------



## TheBlademaster01

I'm guessing 700-800 dollars. Not really sure though


----------



## Svartie

Hi, got myself a Z9PE-D8 WS a week ago and this week i revived CPU and ram.
When i try to start the system the Q-LED is showing 00.
My thought at first was F*** got a dead CPU. Today i started to read more in to problems with this board and it got me thinking and checked the Supported CPU for the board and the Bios versions. Most Intel Xeon E5 v2 only started to be supported in 5103.

So the question. As i cant start the board there is no way i can check what Bios its running but its a rev.1.04 board.
Is there anyone with stock Bios that got a rev. 1.04 that could tell me what version they got? or is there another way of finding out what version its running?
Also is it possible to Flash bios with a CPU that the board cant recognize?

The CPU that im trying to start is Intel Xeon E5-2620 v2.


----------



## TheBlademaster01

I have rev 1.04. It comes with BIOS 0503.

Your best option is to get a pre flashed bios chip from eBay with 5103 loaded on it. Or buy a cheap Sandy chip.


----------



## Svartie

Wow that was a fast replay








Will give both option a go. Ofc the cheaper option first with chip from eBay








Thank you Blademaster01 for the response


----------



## TheBlademaster01

Sometimes you meet the person with the right information at the right time. Glad I could help









This thread is gold, miles ahead of Asus support.


----------



## arvidab

Quote:


> Originally Posted by *Svartie*
> 
> Wow that was a fast replay
> 
> 
> 
> 
> 
> 
> 
> 
> Will give both option a go. Ofc the cheaper option first with chip from eBay
> 
> 
> 
> 
> 
> 
> 
> 
> Thank you Blademaster01 for the response


Ooh, another Sweeede!









Yea, this thread is pretty darn good.


----------



## Svartie

arvidab.


----------



## Lyuokdea

Hi All, I'm having a huge problem here. Trying to install a Noctua NH-U12S cooler onto the board, and using the new Xeon V2 chips -- but I can't get the cooler to screw into the mounting pins, it floats about 1/10th of a millimeter above the pins and won't screw in.

The processor seems to be installed correctly - plastic cap removed, processor layed in fitting triangle to triangle. I've taken off the processor and checked that all the pins align. The processor lays down easily and the motherboard locks it back in with moderate (but normal) force on the locking mechanism.

I install the LGA 2011 adapters to the motherboard, doesn't seem to be a problem. The double screws go into the four outer screw holes on the motherboard, with the longer, thinner screws pointing up. The brackets go on top of those (I've tried both vertical and horizontal orientations) the thumb screws screw in the bracket. Everything looks like the diagram.

BUT, then I can't get the heat-sink to screw into the center screws on the bracket. By tilting the heatsink slightly I can get one screw in, but then getting the other screw in is completely impossible (even with much more muscle than I'd like to put into a $2k motherboard/CPU combo.) I've tried to reset everything with no real luck. Any ideas? Am I doing something dumb - are these parts not compatible for some reason? Are Xeon v2's thicker than old chips?

Ahhh!


----------



## arvidab

Had a quick look at the mounting mechanism for that cooler. Is it possible the center bracket has been flipped up-side-down? Pretty far fetched though if they are new.

1/10mm sounds like it would be possible to compress the springs to get them to begin to screw in. A possible solution would be to put small washers on the 2011 adapters to get it a bit higher up.

I'd be surprised if the v2's were thicker, but I don't know as I'm on old CPU's. I'd measure mine if they were not in my main rig, sorry.


----------



## Lyuokdea

Thanks for the help - the small washer might be a good idea - that would at least let the screw connect and then I can use the springs to get the right amount of pressure.

Just looked at the center bracket - that was a good idea too...However it is in the position where it is curved lower - the problem would be worse (probably by a few mm) if it were flipped the other way).


----------



## arvidab

Quote:


> Originally Posted by *Lyuokdea*
> 
> Just looked at the center bracket - that was a good idea too...However it is in the position where it is curved lower - the problem would be worse (probably by a few mm) if it were flipped the other way).


Ah, thought so. I hope you can get it squared. :/

I know AndyE have had SB-E and is now using IB-E, he should know if they are different thickness or if he had similar problems as you. You could try to send a PM to him.


----------



## anubis1127

Quote:


> Originally Posted by *Lyuokdea*
> 
> Thanks for the help - the small washer might be a good idea - that would at least let the screw connect and then I can use the springs to get the right amount of pressure.
> 
> Just looked at the center bracket - that was a good idea too...However it is in the position where it is curved lower - the problem would be worse (probably by a few mm) if it were flipped the other way).


The screws are typically tight before you install them. I would try screwing it down to just the 2011 mounting adapters. Just mount it to a table, or piece of wood, then apply all the force needed to get it secured down, repeat a few times, maybe leave it on overnight, get it loosened up a bit, then try again on the board.


----------



## Lyuokdea

Thanks, I'll try that for tonight - though it doesn't seem like the springs provide a lot of the pressure at all, in preventing the screws from turning in all the way.

Here are two pictures, which seem to illustrate the problem (and my complete bracket setup).
http://imgur.com/sGdx1mf,CbL6iao#0
http://imgur.com/sGdx1mf,CbL6iao#1

In the one with the heatsink attached - the screws hover just about a 1/10 mm over the screws - but no amount of force (short of tilting the entire heatsink to one side) will let them screw in. It seems that it should be fairly easy to at least start the first screw (and then it should take some effort to screw in the 2nd all the way, and then go back to the first). But I can't even apply enough force to get started.

Thanks again for all your help.

~Lyuokdea


----------



## squidman

Quote:


> Originally Posted by *arvidab*
> 
> Ooh, another Sweeede!
> 
> 
> 
> 
> 
> 
> 
> 
> 
> Yea, this thread is pretty darn good.


Det skulle bare mangle søta bror! (just a shout out to my neighbor there....)....and to anubis....ex-Chevy Chase here....wow small world....


----------



## anubis1127




----------



## squidman

OK While I'm here, more hdd and controller stuff....

My Startech controller has 4 SATA 6 GB/s ports, which I thought I needed for the extra hdd's (at 12 at the moment, space for another maybe 4)....but just realized the onboard marvell controller (now taken by 2xssd's and 2xwdd greens in the hybrid mode) uses the same or similar chip, which allows for hdd port replication (i.e. you find a specialized cable).

Soooo...does anyone have any experience the using port replicator function off of a marvell raid controller? Exactly what kind of cable do you need?

Secondly, backup software...have actually gone back to windows backup (correctly recognizes that my OS and program files are on 2 separate ssd's), after having tried (before this build) Acronis, Paragon, etc etc. Any recommendations? Have had pretty bad experiences with various and sundry....

and to all....thank god for winter coming soon...just a little something from where I used to live....'totally dope' or something....(not a teenager anymore but looks pretty sick.....


----------



## Emji

Hello Everyone,

First of all, i would like to thanks anyone who contributed to this awesome thread. Clearly, there is way more informations here than on the Asus support.

here's my build :

Asus Z9PE-D8 WS (ohh really !?)
2x Intel Xeon E5-2630v2
Corsair VENGEANCE 4 x 8 Gb DDR3 PC15000 CAS 10 (CMZ32GX3M4X1866C10)
EVGA GTX 680 2gb
Cooler Master Silent Pro M2 1000W 80PLUS Silver

At first, I built the setup with the standard BIOS (i think 1.04). obviously there was no post happening (no display, Q-Code 00) as the Ivy-Bridge E are only supported since the bios 5103 (be it with 1, or 2 CPU plugged).
So after ordering a bios chip updated with the lastest Version on Ebay , it could finally post, and i was able to go inside the Bios, but only with 1 CPU. The RAM is perfectly working.

As soon as i plug the second Xeon on the board, there is no post, and the motherboard is left with a Q-code "bC".

I've read on this thread :
Quote:


> Originally Posted by *mateuszgpucomp*
> 
> Hi
> I use Z9PE-D8 WS with 2 Xeon E52696 v2.
> All work greats, Bios 5103, memory if someone wants to use 2 CPU *MUST BE ECC*.
> Non ECC memory work with one CPU (CPU must be in slot CPU1)
> more test of this Xeon


Does anyone else here had the same issue i am having right now, and could fix the problem by switching to ECC memory ?

I don't quite understand why the RAM is working fine with 1 CPU, but not with the 2 of them. the Memory is listed on the RAM support list on the Asus website, technically it should be working. why not ?

Since i already spent way too much money on this build, i would be quite upset to have to spend another 500€ for a kit of ECC memory.

Can we expect Asus to release a new Bios which could possibly fix the issue with non-ECC Memory working with BOTH cpu ?

Thank you.


----------



## squidman

Try reseating your memory first! That cost me 3 days.....it wants to be in there properly, like really jammed in there (without stressing mobo of course)..the board supports non-ecc memory after all. But am on early ES chips, so that's about the limit of my knowledge...









Edit and on V1 chips...so ymmv..etc etc...


----------



## Retell

I'm having the same issue! I haven't tried a single Xeon yet, but I just came from an Intel Barebones W2600CR. I had done the BIOS update and put in 1x 2697 V2, no problem. Putting in the second prevented posting. It worked with 2x 2687W's though. So I moved over to the Asus board hoping it'll fix my issue... Nope I game the same 6C error.









Asus Z9PE-D8 WS
2x Intel Xeon E5-2697 V2
8x G.Skil 8Gb RipJaws Z

Quote:


> Originally Posted by *Emji*
> 
> Hello Everyone,
> 
> First of all, i would like to thanks anyone who contributed to this awesome thread. Clearly, there is way more informations here than on the Asus support.
> 
> here's my build :
> 
> Asus Z9PE-D8 WS (ohh really !?)
> 2x Intel Xeon E5-2630v2
> Corsair VENGEANCE 4 x 8 Gb DDR3 PC15000 CAS 10 (CMZ32GX3M4X1866C10)
> EVGA GTX 680 2gb
> Cooler Master Silent Pro M2 1000W 80PLUS Silver
> 
> At first, I built the setup with the standard BIOS (i think 1.04). obviously there was no post happening (no display, Q-Code 00) as the Ivy-Bridge E are only supported since the bios 5103 (be it with 1, or 2 CPU plugged).
> So after ordering a bios chip updated with the lastest Version on Ebay , it could finally post, and i was able to go inside the Bios, but only with 1 CPU. The RAM is perfectly working.
> 
> As soon as i plug the second Xeon on the board, there is no post, and the motherboard is left with a Q-code "bC".
> 
> I've read on this thread :
> Does anyone else here had the same issue i am having right now, and could fix the problem by switching to ECC memory ?
> 
> I don't quite understand why the RAM is working fine with 1 CPU, but not with the 2 of them. the Memory is listed on the RAM support list on the Asus website, technically it should be working. why not ?
> 
> Since i already spent way too much money on this build, i would be quite upset to have to spend another 500€ for a kit of ECC memory.
> 
> Can we expect Asus to release a new Bios which could possibly fix the issue with non-ECC Memory working with BOTH cpu ?
> 
> Thank you.


----------



## Retell

I can confirm ECC memory is required.

I threw in 16gb of Kingston ECC and it fired right up.
Quote:


> Originally Posted by *Retell*
> 
> I'm having the same issue! I haven't tried a single Xeon yet, but I just came from an Intel Barebones W2600CR. I had done the BIOS update and put in 1x 2697 V2, no problem. Putting in the second prevented posting. It worked with 2x 2687W's though. So I moved over to the Asus board hoping it'll fix my issue... Nope I game the same 6C error.
> 
> 
> 
> 
> 
> 
> 
> 
> 
> Asus Z9PE-D8 WS
> 2x Intel Xeon E5-2697 V2
> 8x G.Skil 8Gb RipJaws Z


----------



## Lyuokdea

Also having an issue with 2x 2670v2. Already had a new bios copy from ebay which is inserted -- but I get no post (and not even any error codes on the board). Everybody else is getting an error code from the motherboard?

I think this might be another sort of error though - I don't even get any fans to run - the system makes a very short sound and the fans move about 2cm and then everything stops. However, the motherboard lights stay on (i.e. to give me the option to restart) -- but there is no other sign of life on the board.

Anything particular I should try? I guess I should go down to one processor and breadboard the machine. I've checked the backplate for extra screws - and I didn't put any in accidentally or anything. I checked that the power supply turns on if pins 15/16 are shorted -- but I don't have a voltmeter to check that the voltages are correct. I have a second power supply (but connected to the working computer I am on now).

Is there any weird connector that most computers don't have that I need to power or something? I thought the MOLEX connection to the motherboard was weird (haven't seen that before) , but I did that.

Thanks,

~Lyuokdea


----------



## thunderthief

Lyuokdea, that sounds like behavior of a board without enough power. There's the big 24 pin power connector, then two 8 pin power connectors for both CPUs. Both those 8 pin plugs and the 24 pin plug need to be connected otherwise the board won't start (at least mine wouldn't). On that note, my system is also built with two E5-2670v2 chips, 4x16GB Kingston EEC RAM at 1600MHz, Corsair AX1200i, and 2x CoolerMaster 120M coolers. The board appears to be powered fine (power, reset button LEDs, a nearby LED, and the BHC LED) all illuminate. But powering on turns everything on except the system doesn't POST, and the numeric LEDs stay at 00. Does the same thing with any combination of RAM, 1 and 2 CPUs, with and without graphics card, etc etc. Right now it's sitting on a bench and it still doesn't POST. I will note that two pins in the first CPU socket got bent incorrectly after seating the CPU initially. They're bent back but to no avail. We have another board arriving on Monday. I'll keep you updated on what was right/wrong. Perhaps we'll be able to help each other out.


----------



## Lyuokdea

Quote:


> Originally Posted by *thunderthief*
> 
> Lyuokdea, that sounds like behavior of a board without enough power. There's the big 24 pin power connector, then two 8 pin power connectors for both CPUs. Both those 8 pin plugs and the 24 pin plug need to be connected otherwise the board won't start (at least mine wouldn't). On that note, my system is also built with two E5-2670v2 chips, 4x16GB Kingston EEC RAM at 1600MHz, Corsair AX1200i, and 2x CoolerMaster 120M coolers. The board appears to be powered fine (power, reset button LEDs, a nearby LED, and the BHC LED) all illuminate. But powering on turns everything on except the system doesn't POST, and the numeric LEDs stay at 00. Does the same thing with any combination of RAM, 1 and 2 CPUs, with and without graphics card, etc etc. Right now it's sitting on a bench and it still doesn't POST. I will note that two pins in the first CPU socket got bent incorrectly after seating the CPU initially. They're bent back but to no avail. We have another board arriving on Monday. I'll keep you updated on what was right/wrong. Perhaps we'll be able to help each other out.


Thanks for the quick response - I'd definitely like some help here, and will let you know if I find anything that works....

Both 8 pin adapters as well as the 24 pin adapter are plugged in. It's possible that the PSU is not providing enough power (or has some sort of fault).

What confuses me is that the motherboard shows on - i.e. green lights are illuminated, but the fans are not spinning -- even the ones that are connected to the MOLEX adapter, and which don't go through the motherboard at all. That seems to me like a PSU problem - i would think if the power-supply is providing power to the motherboard, and if it thinks it is on, it should at least be spinning it's own fans.

On the other hand, it does spin it's own fan if I just short pins 15/16. Of course, it doesn't have to provide any real power then.


----------



## thunderthief

The mobo controls the PSU to turn on and off, so the problem may be with either but given the...temperament of this board, it's probably the board. What is the PSU rated to? I've heard of these boards not powering on due to insufficient power. Also, a friend of mine built a system a while back (with a different board but same chipset) and had your same problem. There are likely several problems that manifest themselves identically, but he wrecked the pins in the socket. Press power, system spins up for a moment, then crash.


----------



## Lyuokdea

That could be it as well. PSU is rated to 1050W, so that's not a problem (I built in some overhead in case I added a Kepler card in the future). I have the GPU in at the moment, but am going to take it out for testing.

I had some problems applying the heatsink (Noctua thinks the screws might have been slightly too short) - and so applied a lot of pressure to the board. I don't think anything got bent, but it's hard to know for sure.


----------



## Lyuokdea

Hey - situation has improved.

Took out all the RAM and the GPU. Booted, fans are running and system now shows an error code "60" = "DXE Core is Started"

Obviously I expected an error code sans RAM/GPU - so it's something that the fan is running and I'm getting to error codes.


----------



## thunderthief

What GPU and RAM are you using?


----------



## Lyuokdea

Ram is 4x Samsung DDR3-1600 16GB/1Gx4 ECC/REG CL11 Samsung Chip Server Memory (D316GR16G1)

GPU is Galaxy Geforce GTX 580.

Just inserted two of the 4 RAM sticks, still boots and gets power, error code bounces around now ends at 62. Going to try to seat the GPU and see if I get a boot.


----------



## Lyuokdea

Ok - so then I add the graphics card in, and the system goes back to "no boot whatsoever". There's a slight click and then everything instantly turns off. Do people think this is a power-supply issue, some sort of odd motherboard issue (maybe affecting the PCI-E slot?) Or could a graphics card issue cause this.

Graphics card looks like it is seated alright - it's a bigger card than I'm used to - but I don't see any exposed metal on the pins, and the plastic clip is locked in. I've switched which PCI-E port I am using on the power supply.

Just to check, this is a 8+6 pin GPU, the yellow "+12V" on the GPU should be up (closer to the backplate) correct? And for the 6 pin, the two pins that I leave off from the normal 8 pin adapter are the two grounds that are on the right side, correct?


----------



## Lyuokdea

Wait - two dumb questions. What is the VGA Controller setting on the motherboard? Does that need to be disabled in order to attach an extra card?

Secondly, I have the GPU in on PCI-E port 3 (which is the second PCI-E x16 port. Nothing else is plugged into any other PCI ports. That is ok right?


----------



## Lyuokdea

Success - sort of! Adding an old GPU (a 9600 GSO) makes the system POST correctly (goes to BIOS and sat there stably for about 5 min). I powered it from the same socket of the power supply and put it in the same PCI-E port. I think that rules out the PCI-E port being the problem.

So is the GPU bad, and could that actually make the system power off immediately? I don't really see anything else it could be - though I guess in theory I'd like to test another high end card to make sure there isn't some lack of power problem. It's a 8+6 pin GPU - so it requires two different PCI-E power connections - what if I draw that from two different lines of the power supply? The power supply I have now, has PCI-E ports that split to an 8+6 port, so I've been plugging both into the GPU from the same line.


----------



## x86box

How powerful is your PSU? I couldnt use anything under 1200watt, I eventually settled on the 1600watt LEPA. Simply because it has the amerage amount needed on 3.3V and 5v for CPU. Check that your PSU has enough power 5v and 12v amperage for GPU.


----------



## Lyuokdea

1050W -- that really should be plenty, This thing shouldn't be drawing more than 500 in the current configuration (115W/processor + 20W for the MB + 4 x 10W for RAM + 225W or so for the GPU).


----------



## x86box

I think its your PSU, I have only used one PSU under 1400watt and I had major problems. I blew up 1 motherboard and 1 CPU using a 1000watt PSU. I went to the 1200 antec and same problems. It would start up and instantly turn off. When I disabled 1 CPU it would boot up normally. Then changed the PSU and all was normal again. i went to 1600 Lepa and not a single issue.

I'm using x2 2680, Nvidia Titan, 64GB Corsair 1600mhz, and x12 3Tb drives, water cooling pump, + 7 fans, and slots full of Atto Raid / HBA / BlackMagic Decklink / 10Ge Myricom. But most of that wasnt in during the build of course.

Its more about the available 12v amperage and how spread out it is. I wasnt getting enough amperage on CPU outputs, thats why it blew.


----------



## ZDngrfld

Quote:


> Originally Posted by *x86box*
> 
> I think its your PSU, I have only used one PSU under 1400watt and I had major problems. I blew up 1 motherboard and 1 CPU using a 1000watt PSU. I went to the 1200 antec and same problems. It would start up and instantly turn off. When I disabled 1 CPU it would boot up normally. Then changed the PSU and all was normal again. i went to 1600 Lepa and not a single issue.
> 
> I'm using x2 2680, Nvidia Titan, 64GB Corsair 1600mhz, and x12 3Tb drives, water cooling pump, + 7 fans, and slots full of Atto Raid / HBA / BlackMagic Decklink / 10Ge Myricom. But most of that wasnt in during the build of course.
> 
> Its more about the available 12v amperage and how spread out it is. I wasnt getting enough amperage on CPU outputs, thats why it blew.


I'm running my Z9PE on a 650 watt Seasonic...


----------



## anubis1127

I'm using a 620W Seasonic, or I was before I took apart my folding rig. When I put it back together I may use a Seasonic x750 I have.


----------



## x86box

Which CPU and GPU are you using? I tried 1000w and 1200w versions and would not start up with x2 2680 and Nvidia Titan OR 680 4GB. Nothing else in the board. Then swappd it for a 1600 and it worked great.

But I do have some 6 core 2.3ghz that work just fine off a 800watt, in my other Z9pe Rig.


----------



## anubis1127

Quote:


> Originally Posted by *x86box*
> 
> Which CPU and GPU are you using? I tried 1000w and 1200w versions and would not start up with x2 2680 and Nvidia Titan OR 680 4GB. Nothing else in the board. Then swappd it for a 1600 and it worked great.
> 
> But I do have some 6 core 2.3ghz that work just fine off a 800watt, in my other Z9pe Rig.


I'm using 2x 2680s, and a hd5450, haha, so that is the difference.


----------



## sagitarius2k

Quote:


> Originally Posted by *x86box*
> 
> Which CPU and GPU are you using? I tried 1000w and 1200w versions and would not start up with x2 2680 and Nvidia Titan OR 680 4GB. Nothing else in the board. Then swappd it for a 1600 and it worked great.
> 
> But I do have some 6 core 2.3ghz that work just fine off a 800watt, in my other Z9pe Rig.


wow, is that titan really hunger for power? i planed to use mine rig with r9 290x CF or GTX 780i SLI. Should 1200 watt corsair 1200AXi is enough for this?


----------



## Lyuokdea

Still, a GeForce 580 is rated at 244W Maximum Power.

http://www.geforce.com/hardware/desktop-gpus/geforce-gtx-580/specifications

There's no way that a GeForce 580 and 2 115W 2670v2's are overwhelming a 1050W power supply. Unless perhaps there is something about total power on a single rail, or if the power supply is faulty.

Edit: From the specs the Seasonic has a single 12V rail (all the outputs must be connected) - and is capable of supplying up to 87A (1044W) - so I don't think that's the issue either.

http://www.tweaktown.com/reviews/4508/seasonic_x_1050_1050w_power_supply_review/index.html


----------



## x86box

Well, try this.

Take out your 2nd CPU.

Put the 580 in.

If it doesn't boot it's the GPU. If it does boot you need a different PSU. The CPU + GPU could be more than your psu can handle. Simple test.

I encountered this with 2680. The 1k and1.2k did not supply enough to CPU. Didn't matter how much power I was actually going to use in the rest of the system.


----------



## squidman

Quote:


> Originally Posted by *PhxTriode*
> 
> Thank you for the reply, I just picked up some H80's and should be here tomorrow. Hope I can get them to fit


Have 2x h80i's and have a had a world of trouble....suffice to say, they don't work connected via corsair link to the commander. However, finally got them to work using nzxt internal usb header expansions (x2) as there are only 2 usb internal headers on my z9pe-d8 ws. System is actually stable and working...albeit have updated all firmware for h80i's, commander and cooling node. Corsiar link cable from node to commander works, just not the rest. Once it's setup, brilliant...but it took about of month of cussing and disassembling and ranting on corsair forums....























Link to the nzxt usb header...absolute lifesaver....(or you can use up the remaining usb internal headers on your mobo...)..http://www.nzxt.com/product/detail/99-iu01-usb-expansion-case-accessory

Other...the fans that come with the h80i's are really noisy...I would seriously consider some noctua's iff you have the extra dough....









Edit they actually work pretty well....rendering with 2 ES 2665's near 90%, temps never go above 45c on quiet mode.....but anything above quiet mode is pretty loud....







Of course they are only 2.3 ghz chips..


----------



## Retell

Quote:


> Originally Posted by *squidman*
> 
> Have 2x h80i's and have a had a world of trouble....suffice to say, they don't work connected via corsair link to the commander. However, finally got them to work using nzxt internal usb header expansions (x2) as there are only 2 usb internal headers on my z9pe-d8 ws. System is actually stable and working...albeit have updated all firmware for h80i's, commander and cooling node. Corsiar link cable from node to commander works, just not the rest. Once it's setup, brilliant...but it took about of month of cussing and disassembling and ranting on corsair forums....
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> Link to the nzxt usb header...absolute lifesaver....(or you can use up the remaining usb internal headers on your mobo...)..http://www.nzxt.com/product/detail/99-iu01-usb-expansion-case-accessory
> 
> Other...the fans that come with the h80i's are really noisy...I would seriously consider some noctua's iff you have the extra dough....
> 
> 
> 
> 
> 
> 
> 
> 
> 
> Edit they actually work pretty well....rendering with 2 ES 2665's near 90%, temps never go above 45c on quiet mode.....but anything above quiet mode is pretty loud....
> 
> 
> 
> 
> 
> 
> 
> Of course they are only 2.3 ghz chips..


Glad to hear


----------



## Emji

Quote:


> Originally Posted by *Retell*
> 
> I can confirm ECC memory is required.
> 
> I threw in 16gb of Kingston ECC and it fired right up.


great ! that is 1 more confirmation :>

So i contacted the Asus support again to let them know about this issue related to E5-v2 serie CPUs and non-ECC ram and that some of you guys could make it POST with ECC memory.
After convincing the support agent that it was completely useless to send the board in RMA, she forwarded my ticket upper in the support hierarchy. Hopefully we will have some good news soon? maybe a new BIOS in the upcoming weeks ?







let's cross our fingers !


----------



## thunderthief

My first build failed. Here's what happened:

Install two E5-2670v2's in each socket (inadvertently bent two pins in CPU socket 1 but those are easy enough to fix)
Install the RAM (Kingston EEC DDR3 1600 MHz)
Install the mobo to the case
Install Corsair AX1200i PSU
Install two Cooler Master 120M's (fans setup to pull)
Install AMD 5760? Radeon (cannibalized from a Mac Pro








Wire everything up like a champ with some 4TB HDDs
Turn the PSU on and...more lights than there should be...

The power and reset switch LEDs were ON, as was a third LED nearby the bottom right corner and the BMC LED near the bottom leftish edge. Now that I know better, that was the first sign of trouble. Hitting the power button activated everything (from case fans to GPU) but alas, the little workstation was a vegetable. The status LEDs stayed at 00 and would occasionally flicker. I sequentially removed everything (including the mobo) until the only things left were a CPU and a lonely stick of RAM. Still nothing. The board was dead, and my research group's wallet got a bit lighter...

Yesterday the second board arrived. This time it was bench tested while sequentially inserting every component. Success. Turning on the PSU only illuminated the power LED, and the status LEDs happily blither on while POSTing. The first problem? Probably something annoyingly minor. I've learned that the simple answer is often the correct one; and sometimes a dead board is simply a dead board. There may be one innocuous component that has failed, but it's enough. Hooray for engineering standards, but I digress.

I'm happy to answer whatever questions I can.


----------



## mking

thunderthief - but what bios did your first board have ? sounds like it didn't have the v2 5103 bios and the second board did. Mike


----------



## thunderthief

No clue. The first is packed up and going back to superbiiz.com. The second board came from Amazon. If you can figure that out from the serial number, board 1 starts with D7S1. Board 2 is D9S1.


----------



## Donkey1514

I'm going to try to get the OP switched to me so i can post some relevant info in it concerning this board


----------



## anubis1127

That would be most excellent. Recommended BIOS for different ES chips, recommended RAM, and PSUs, etc would all be good stuff to have in the OP instead of digging through 800+ posts for pertinent info.


----------



## tomyboy

Anyone else getting blue screens in Windows 8 with different messages memory, watchdog, etc?
Always when renddering. Sometimes at 100% load, sometimes with no load at all, just looking at the project.
Really close to do samo physical damage to the board with some High Volts and clame RMA and just buy something else. This board is really my worst purchase.


----------



## Svartie

Quote:


> Originally Posted by *thunderthief*
> 
> No clue. The first is packed up and going back to superbiiz.com. The second board came from Amazon. If you can figure that out from the serial number, board 1 starts with D7S1. Board 2 is D9S1.


What version do the board say it is? If its a rev.1.04 its bios that can't understand how to use a Ivy chip. You can find this info middle of the board next to the ram slots.
Quote:


> Originally Posted by *TheBlademaster01*
> 
> I have rev 1.04. It comes with BIOS 0503.
> 
> Your best option is to get a pre flashed bios chip from eBay with 5103 loaded on it. Or buy a cheap Sandy chip.


bios chip cost me like 9 euro. Everything now works just fine.


----------



## Emji

to anyone interested, this is the BIOS nightly build (5206). they told me that is was supposed to support 2630v2 which is not listed as compatible CPU (yet it is working fine with 1 CPU...) but they didn't say anything about the issue with non-ECC memory. i'll update and see how it's going.

here's the link :
http://dlcdnet.asus.com/pub/ASUS/mb/LGA2011/Z9PE-D8-WS/BIOS/Z9PE-D8-WS-ASUS-5206.zip


----------



## Emji

OK guys, after updating to 5206, I am now booting on my 2 Xeon E5-2630v2 with 4x8gb of non-ECC memory.


----------



## mking

@Emji, awesome news - just need my ECC ram to arrive now so I can post and flash from 5103 to 5206








I can smell success just over the horizon...

Also just general FYI there is no matching between board rev and bios rev. I got 1.04 board rev and 3xxx bios.

Mike


----------



## psyq3212

5206 fixes incorrect timing detection for ECC RAM with IVB-EP, so it looks like ASUS spent some time fixing memory detection/training issues.

Also, they now correctly set Isoch to "disabled" for 12-core CPUs, and also add additional IvyTown options.


----------



## mking

Yes I can confirm with 5206 I have 2 x 2687W V2 running with 64GB (8 X 8) of dominator GT @ 1866. 89.25GB/s and running prime95 torture test quite happily.

I continue to be amazed by how cool the SB and IB Xeons run. I have H90 cooling on these and with torture test I still can't get the CPU over 50C !! Very disappointing to not be able to wind up the fans with a big compute









Txs to you all for helping me get through this.

Hopefully plain sailing from here on....

Mike


----------



## mking

ok 5206 is now official bios update http://support.asus.com/Download.aspx?SLanguage=en&p=1&s=42&m=Z9PE-D8%20WS&os=8&ft=3&f_name=Z9PE-D8-WS-ASUS-5206.zip#Z9PE-D8-WS-ASUS-5206.zip


----------



## andrefsf

Hi guys!

This baby is helping us running some CUDA enabled applications in Linux. Great motherboard! Overall the layout is very good, but the headers on the bottom (usb2.0, usb3.0 and serial) should be right angled. If you are using a 4 GPU configuration is no easy to connect the last GPU...




Regards


----------



## sagitarius2k

Quote:


> Originally Posted by *andrefsf*
> 
> Hi guys!
> 
> This baby is helping us running some CUDA enabled applications in Linux. Great motherboard! Overall the layout is very good, but the headers on the bottom (usb2.0, usb3.0 and serial) should be right angled. If you are using a 4 GPU configuration is no easy to connect the last GPU...
> 
> 
> 
> 
> Regards


hello, can you post your entire pc photo in here? i am using 900d to, and still strugling about fans









oh, how much temp do you get with those CPU cooler? and how much is your room temprature? thanks


----------



## andrefsf

The fan configuration I left it as default, 3X120mm input and 1X140mm output. Regarding the temperatures the ambient is around 22ºC, the CPUs (intel Xeon E5-2620) are idling at 37-40ºC and when I run a calculation they go for the ~50ºC.


----------



## Rydland

Hey Guys,

any updates from you on the new 5206 bios ?
Do you know if it supports non-ECC ram for 2630v2 ?









Cheers,

Rydland


----------



## Emji

Quote:


> Originally Posted by *Rydland*
> 
> Hey Guys,
> 
> any updates from you on the new 5206 bios ?
> Do you know if it supports non-ECC ram for 2630v2 ?
> 
> 
> 
> 
> 
> 
> 
> 
> 
> Cheers,
> 
> Rydland


Yes it does,

i am running with 4x8gb of non-ECC ram with 2x 2630v2, it's working like a charm...


----------



## Rydland

Quote:


> Originally Posted by *Emji*
> 
> Yes it does,
> 
> i am running with 4x8gb of non-ECC ram with 2x 2630v2, it's working like a charm...


Thats really great to hear







Thanks for the quick reply!


----------



## Rydland

Hey again guys,

first of all, awesome forum! Finally i got around to make myself a user









Maybe some stupid questions here in the beginning, hope you will bear with me.

Tomorrow i am getting my 2 x e5-2630v2 CPU's, Z9PE-D8 WS motherboard and 32gb corsair dominator ram.
From what i can read in here, i wont be able to boot with this config. unless i have the newest BIOS. Since that came out wednesday, i am definately not getting that.
Would it be possible for me to boot with just one of the CPU's and 1 stick of ECC ram ? Saw some of you guys had luck with some other v2 Xeon's. Is there any way of updating BIOS without a CPU in it ? (Really stupid question i guess, but... Well, gotta try all options







)
I wouldnt be able to use a 3930k to boot with would i? Guess the motherboard only support Xeon CPU's.
Some of you mentioned that you could buy a new Bios Chip. Not really sure i am up for that. My techincal skills isnt all that good









If you guys have any input at all, it would be grately appreciated.

Cheers,
Rydland


----------



## anubis1127

Assuming your board has a new enough bios that supports the v2 chips with ECC RAM, running one CPU and a stick of ECC should work.

Swapping the BIOS chip on this board is not bad at all. You do have to be careful I suppose, but basically you just pull the old one out, and put the new one in.


----------



## arvidab

Quote:


> Originally Posted by *Rydland*
> 
> Tomorrow i am getting my 2 x e5-2630v2 CPU's, Z9PE-D8 WS motherboard and 32gb corsair dominator ram.
> From what i can read in here, i wont be able to boot with this config. unless i have the newest BIOS. Since that came out wednesday, i am definately not getting that.
> Would it be possible for me to boot with just one of the CPU's and 1 stick of ECC ram ? Saw some of you guys had luck with some other v2 Xeon's. Is there any way of updating BIOS without a CPU in it ? (Really stupid question i guess, but... Well, gotta try all options
> 
> 
> 
> 
> 
> 
> 
> )
> I wouldnt be able to use a 3930k to boot with would i? Guess the motherboard only support Xeon CPU's.


Booting with just one CPU, you can use non-ECC RAM afaik, if it's on at least BIOS 5103. Don't know about the 3930k, it wight work, if you happen to have one it couldn't hurt to try it.

Quote:


> Originally Posted by *Rydland*
> 
> Some of you mentioned that you could buy a new Bios Chip. Not really sure i am up for that. My techincal skills isnt all that good


Thankfully the BIOS chip is in a socket on these motherboards, an approach more manufacturers should employ. There are also tools for for pulling the chip out of it's socket but it can be done without. It's really not that difficult, take it easy and make sure that you install the chip in the right direction.
Just be careful when you do it, there are a number on Youtube you can watch to get a feeling for how it works.

5103 supports v2 chips but not non-ECC RAM with two CPU's installed, those BIOS chips can be found on eBay for 10-15 bucks, make sure you ask the seller which version it is on it before buying though.

There are also other methods of flashing a BIOS, but simply pulling the old chip and installing a new one is what I see as easiest.


----------



## Donkey1514

OP UPDATED!


----------



## ciceu4

Hello guys i need an advice. 2 days ago I flash the last bios and the motherboard did not start and I get the code 62. Today I borrow a gpu pci 2.0 and today I boot normally so I tried to flash an older bios to have support for gtx 6xx .And now is not working at all and I get code 14. I have 2 Es chips. What I can do ? Why my sistem worked with the latest bios and when I flash 5003 just died , I try using the pci 2.0 gpu but is stuck and get code 14.thanks in advance .


----------



## anubis1127

Which ES chips do you have? I know some of them are only compatible with certain BIOS, example, if they are B0 then you can't flash beyond the 0703 BIOS, and if c0 I think you can't flash beyond 3506. That is for SBe, I am not sure on IBe if you have that, I know you must start at 5103, but don't know about compatibility.

Have you tried doing all the normal things like power off, clear CMOS, remove CMOS battery, etc?


----------



## ciceu4

I do not know the cpus stepping , are sandy bridge. But I had 3506 bios until 3 days and had only problems with gpu pci 3.0 . So I decide to update to 5206 but the pc didn't start with a gpu pci3.0 and work perfect with gpu pci 2.0. So I flash back 5004 and is not working at all. I try clear cmos and I remove ram an try all the pci slots with gpu 3.0 and old gpu 2.0 . From what I read I think my only chance is to buy a bios chip. I do not understand why my board worked with first bios and other update 3506 and 5206 and is not working with 5004. Maybe I can borrow a cpu an upgrade again to 5206 and buy another gpu card.I can not believe that such a expensive card has a lot of problems with pci 3.0 and no support . Before upgrade my gpus I used this card for 1 year without any problems.I have this motherboard since august 2012.


----------



## psyq3212

@ciceu4,

Can you tell us the QDF code of your CPU (it should be something like: Qxxx - for example: QB7R or QA92 etc.)

You can see the code by removing the cooler - it is written on the CPU itself.


----------



## ciceu4

The code is QA8X.


----------



## anubis1127

Quote:


> Originally Posted by *ciceu4*
> 
> The code is QA8X.


That explains it, those are B0 stepping CPUs. They will not work with the 5000 series BIOS, I would flash back to 3506. That also explains your issues with newer GPUs, B0 chips don't work with them.

If you want support for modern GPUs, you must get later stepping chips, or ideally retail CPUs, as this current issue you have stems from using unsupported CPUs.


----------



## OwenBurnett

I'll also soon be an owner of one of this boards,
can you answer me a few simple questions upfront.

The Sata ports that are connected to the chipset are gruped in
2, 4 and 4

the manual says an all of this grupes I can make a raid 5
so i guess at least the first 2 and 4 ports belong together.
but what about the 4 that are describes as SCU.

Can I make a raid 5 of all 10 ports or are the other 4 despite also being connected to the chipset seomhow separated?

Cheers
Owen


----------



## ciceu4

Thank you. I will buy another bios chip with version 3506 and try to change the cpus.


----------



## psyq3212

If you are stuck with B0 stepping, PCIe 3.0 will not work with Kepler generation NVIDIA cards (I don't know about AMD, though).

In addition, you'll probably have to put the card in the third PCIe slot, as it will mostly not boot when it is in PCIe slot #1.


----------



## Donkey1514

Quote:


> Originally Posted by *psyq3212*
> 
> If you are stuck with B0 stepping, PCIe 3.0 will not work with Kepler generation NVIDIA cards (I don't know about AMD, though).


anyone have info on which AMD cards work?


----------



## squidman

Quote:


> Originally Posted by *anubis1127*
> 
> That explains it, those are B0 stepping CPUs. They will not work with the 5000 series BIOS, I would flash back to 3506. That also explains your issues with newer GPUs, B0 chips don't work with them.
> 
> If you want support for modern GPUs, you must get later stepping chips, or ideally retail CPUs, as this current issue you have stems from using unsupported CPUs.


Have an 'older newer' GPU and same ES B0 stepping, my gtx 770 (aka gtx 680) will only work on pci-e 2.0, not the blue ones. Dunno if it makes a HUGE difference, but annoying. Has worked intermittantly (on pci-e 3), but not reliably....so it's back to the black pcie slots for me. I forget the bios, but is the latest one that supports B0 stepping, and seems stable (after the update jumping through hoops business).

Bummer...but I DID pay 600 bucks for the e5-2665's instead of (go newegg!)....







gotta love the flea-bay!

Hmmmm, lots of good deals there now on es chips, including e5-2687w's.....just need to win lottery...


----------



## anubis1127

Quote:


> Originally Posted by *squidman*
> 
> Have an 'older newer' GPU and same ES B0 stepping, my gtx 770 (aka gtx 680) will only work on pci-e 2.0, not the blue ones. Dunno if it makes a HUGE difference, but annoying. Has worked intermittantly (on pci-e 3), but not reliably....so it's back to the black pcie slots for me. I forget the bios, but is the latest one that supports B0 stepping, and seems stable (after the update jumping through hoops business).
> 
> Bummer...but I DID pay 600 bucks for the e5-2665's instead of (go newegg!)....


Ah, good to know. I thought they just did not work. PCIe 2.0 x16 is still fine for a single GPU.


----------



## squidman

Quote:


> Originally Posted by *OwenBurnett*
> 
> I'll also soon be an owner of one of this boards,
> can you answer me a few simple questions upfront.
> 
> The Sata ports that are connected to the chipset are gruped in
> 2, 4 and 4
> 
> the manual says an all of this grupes I can make a raid 5
> so i guess at least the first 2 and 4 ports belong together.
> but what about the 4 that are describes as SCU.
> 
> Can I make a raid 5 of all 10 ports or are the other 4 despite also being connected to the chipset seomhow separated?
> 
> Cheers
> Owen


I have the last SCU ports raided in raid 5. Then the marvell chipset (the four sata 6gps below the top intel ons) in hyper-duo mode. The 6 top can either be controlled by LSI raid controller (which I never got to work) or intel controller. I was not able to get the first 2 (black) 6gps ports to work raided without raiding the rest....so it's all or nothing....the top 6 seem to be either JBOD or RAID, but not a combination. Bummer.. (wanted to raid my top 2 ssd's there). Also setting up the intel thing involves lots of mashing of ctrl+I, both for the scu setup and the intel raid for the top 6.
Note this is for the z9ped8 WS! Have an additional raid card with the same marvell controller, that worked pretty ok, no conflicts..set and forget, just showed up in the marvell software..unfortunately, although both the marvell chips (mobo chipset and raidcard chipset) support port mulitiplier, (you can attach 4 hdd's to one sata port), no way to do this internally....of course with 12? hdd's and 4 ssd's I don't actually have space for many more...maybe 3....








dude, can't have too many hdd's! My raid 5 array with 10 GB is already getting filled up (was 4x4 TB WD RED). Now actually backing this up off-site/online is another matter...


----------



## OwenBurnett

I saw that the E5-2650 V2 have a all core turbo of 500 mhz,
does that mean that i can run them permanently at 3,1 ghz, and if so what would i have to change in the bios?


----------



## anubis1127

It should automatically when the system load dictates it to. If you want to run it 3.1Ghz on all cores, all the time, with no load, you could probably set the max multi to 31, and disable EIST in the BIOS.


----------



## PointFloat

Hi,

I seem to have an odd network problem with my color grading machine, the specs are:
Asus Z9PE-D8 WS
e5-2687w x 2
Corsair-Vengeance-8GB-1600MHz-CL10-DDR3 x 4
Nvidia GT120
Gainward-GeForce-GTX-690-4GB
boot 128GB SSD,
4TB HDD
7x2TB Raid0
Windows 8.1 Pro

The machine connects with PC's fine, yet the problem is with smb connectivity from OSX computers. Sometimes it connects with the finder syntax smb://name, yet mostly results in an "unable to connect" error. When I specify a target drive, e.g smb://name/P, it mounts the drive, yet copying files results in error codes -8084 and -36. The same happens when it sometimes connects with smb://name, no file exchange possible, also tried with IP address instead of the machine name, yet to no avail. The OSX machines are running snow leopard and ML. Now, when connecting from a Mavericks machine, it connects just fine and is enable to exchange files. The snow leopard and ML machines connect fine to other windows 7 and 8.1 machines, hence why it seems like it is a problem with the Z9Pe-D8 WS (Same problem under Win7). Also tried a PCIex ethernet card, which gave same results. Currently running Bios 3109

Any ideas?

Thanks


----------



## poulk

hello again guys!
i need quick help.
i want buy 2 pcs of
INTEL XEON E5 2695 2697 V2 IVB-E 12 CORE 2.5-3.0Ghz B1 stepping
or
INTEL XEON E5 2695 2697 V2 IVB-E 12 CORE 2.5-3.0Ghz B0 stepping
from ebay.
which one i must prefer to not have a problems with asus mb ?
big thx!


----------



## Donkey1514

Quote:


> Originally Posted by *poulk*
> 
> hello again guys!
> i need quick help.
> i want buy 2 pcs of
> INTEL XEON E5 2695 2697 V2 IVB-E 12 CORE 2.5-3.0Ghz B1 stepping
> or
> INTEL XEON E5 2695 2697 V2 IVB-E 12 CORE 2.5-3.0Ghz B0 stepping
> from ebay.
> which one i must prefer to not have a problems with asus mb ?
> big thx!


B1 stepping is newer but doesn't guarantee that it will be 100% compatible.


----------



## poulk

Quote:


> Originally Posted by *Donkey1514*
> 
> B1 stepping is newer but doesn't guarantee that it will be 100% compatible.


are you 100% sure b1 not fit?


----------



## Donkey1514

It'll "fit' but you may encounter Asus dropping bios support for early ES steppings like they did with B0/B1 stepping E5 v1 chips. You may also have incompatability issues with GPUs depending on the bios version that supports those early steppings.

The chips will work but don't count on any support from Asus


----------



## psyq3212

There are no similarities between SNB-EP B steppings and IvyTown B steppings.

I strongly recommend >not< buying B0 IvyTown-s despite the fact that they will work "most of the time". B1 is a different story, but if you can - try to get C0 (QS) or, at least, B3 stepping (which is 4S only, but nothing is stopping you from using it in 2S configuration).


----------



## TheBlademaster01

What is the difference between B0, B1 and C0?


----------



## poulk

so
INTEL XEON E5 2695 4650 V2 IVB-E 12 CORE 2.4-2.9Ghz B3
be best choice ?(there is no C0 on ebay)


----------



## Donkey1514

Quote:


> Originally Posted by *TheBlademaster01*
> 
> What is the difference between B0, B1 and C0?


B0 is usually the earliest stepping for an ES chip and as the testing process matures there may be slight microcode changes for each stepping until Intel reaches what it deems "retail", which is usaually C1 or C2.


----------



## psyq3212

Actually, A0 is the earliest stepping - which in most cases never goes out to customers because it can be unfused and I doubt these would see the light of the day outside the vendor's facilities. First samples to reach customers are A1 steppings (ES0) in many cases, but sometimes it starts with B0. Earliest samples are tested in a very limited sense and can have lots of issues but, as evidenced by the amounts of those available illegally on Ebay, it is pretty clear that Intel became damn good in their fabrication process since these chips are obviously good enough to be sold on Ebay by those ES-chip sellers.

The differences between B0, B1/B2 and B3 are in the number of resolved issues. Naturally, B0 has the longest list of issues since it is very early silicon. As the steppings increase, so does the number of tests performed on them.

The reason why IvyTown B-samples seem to be better in terms of compatibility is probably due to the fact that IvyTown is a "tick" generation, so many problems were resolved in the previous "tock" (Jaketown or SandyBridge EP).

@poulk, yes, if there is no C0, then B3 is the next best choice since it is the most mature of B-steppings. You can even buy 4 of them if you want and put them in a 4-socket motherboard for 48-core goodness







But, since this is an ES chip usual warnings still apply: it might be buggy for what you are trying to do and, of course, to get them on Ebay somebody in the process obviously had to break the NDA with Intel, which means these chips are sold illegally - so do not count on any support from Intel or motherboard vendor.


----------



## Rydland

Hey Guys,

some quick questions i hope you can help me with.

Can anyone confirm that it would work, upgrading the BIOS with a 3930k CPU in the motherboard? I know it isn't on the supported CPU's list, but heard that it would work.

If you upgrade the BIOS yourself, would you do as in this guide?:
http://support.asus.com/Documents/detail.aspx?SLanguage=en&p=1&m=Z9PE-D8%20WS&s=42&hashedid=n/a&os=&no=1704

Can anyone post a link to the BIOS chips from amazon you're all talking about ? With the latest BIOS possible. Can't seem to find it








Preferably from the british site, if there is any sellers.

Hope you can help me out







would really appreciate it!

Cheers,

Rydland


----------



## DualFiz

Wow! Very happy to find this thread!

I'm in the process of a new build and planned to use an Asus Z9PE-D8 motherobard... BUT, after reading all the negative reviews at newegg, I'm a bit cautious.

Coming from the users of this board, in this thread, should I be concerned? Or are the newegg reviews just full of uneducated nubs who didn't plan their build right?

...as for the rest of the build, I will be running a set of dual ES E5 v2 CPU's... "L1" stepping. Any advice on memory or which bios might be most compatible?

Thanks!!!


----------



## DualFiz

Haha, well, this is probably good news. The fact that this thread has become so quiet and well, "dead", suggests that most of the stability issues originally seen with this motherboard have been hammered out, and as it sits now, requires very little involvement or interaction on the part of the owners.

So, I believe this is a good thing!


----------



## TheBlademaster01

I think all memory functions properly with the latest BIOS version on V2 processors.

The board can be buggy but if you got a properly functioning board it's really great with all of the functionality. The newegg results are pretty accurate though, definitely not noobish.


----------



## Rydland

Hey again guys,

sounds good that it is working for all of you. Awesome









I have some stuff i hope you can help me with.
I assembled my build last night and i have problems!
The setup consists of:
2xE5-2630v2
Z9pe-d8 WS
32gb corsair dominator ram.
780gtx
Seasonic 1000w platinum

When i turn it on it starts all the fans and everything and then turns off again after 2-3 seconds. Then it turns on again after being off for 4-5 seconds, and this loop just continues. On, off, on off....
The LED code doesn't show anything.
I purchased a new BIOS chip, that should have the newest BIOS(double checked with the seller). It is plugged in accordingly.
Has anyone tried this before, and do you have any ides of what it could be ?

Hope you can help, would really be appreciated









Rydland


----------



## ZDngrfld

The board will do that if you don't have both 8-pin EPS plugged in. Or it could need to be RMAed. Mine did the same thing. An RMA solved it.


----------



## Rydland

Thanks for the quick answer







. Both 8pins are in. You dont need to plug the 4pin molex EZ-plug in, right?

Well, damnit. I might just need to RMA it.


----------



## ZDngrfld

Quote:


> Originally Posted by *Rydland*
> 
> Thanks for the quick answer
> 
> 
> 
> 
> 
> 
> 
> . Both 8pins are in. You dont need to plug the 4pin molex EZ-plug in, right?
> 
> Well, damnit. I might just need to RMA it.


Nope. Don't need the molex... Sounds like the classic case of needing an RMA unfortunately


----------



## DualFiz

Hmm... well, thats not good (re: Rydland)

I'm really wanting to buy and use this board, but it sounds like it's terrible... literally, according to the internets, more than half of these boards come from the factory DOA. That is totally unacceptable.

I don't want to go supermicro, but it sounds like it's a much better idea. :/

BTW: Good luck with your RMA Rydland, hopefully it's a simple fix!


----------



## sagitarius2k

hello all, i just assamble this case with mine xeon e5 2690

I got problem, after everthing is setup, everything is plug into PSU, my board wont start up. LED start are up, but neither CPU fan nor PSU are start. i press start and nothing happen. Not CPU fan are up or everything else.

Something wrong with my board? or i just miss something to turn this mobo up?

thanks

-edit-

when i start it, q code is not displaying too.


----------



## beaker7

Quote:


> Originally Posted by *sagitarius2k*
> 
> hello all, i just assamble this case with mine xeon e5 2690
> 
> I got problem, after everthing is setup, everything is plug into PSU, my board wont start up. LED start are up, but neither CPU fan nor PSU are start. i press start and nothing happen. Not CPU fan are up or everything else.
> 
> Something wrong with my board? or i just miss something to turn this mobo up?
> 
> thanks
> 
> -edit-
> 
> when i start it, q code is not displaying too.


If board LEDs are lit but you wont even get it to attempt to post it sounds like a short somewhere. Check your power cabling and make sure no extra motherboards standoffs are stucking out, touching the board.


----------



## anubis1127

Yeah, I would double check for shorts. I had the same issue when I put my z9pe-d8 WS in its case the first time. Took it out of the case, booted up fine again. Then I put it back in a few weeks later, being extra careful, and all was well.


----------



## sagitarius2k

Quote:


> Originally Posted by *beaker7*
> 
> If board LEDs are lit but you wont even get it to attempt to post it sounds like a short somewhere. Check your power cabling and make sure no extra motherboards standoffs are stucking out, touching the board.


i am using corsair 900d

when that occured, i took off my motherboard and place it upper it ( board ) case. And still not luck.

the LED i talking about is, start LED and reset LED. But n q code is on. Do Debug code need to turn on?


----------



## sagitarius2k

Quote:


> Originally Posted by *anubis1127*
> 
> Yeah, I would double check for shorts. I had the same issue when I put my z9pe-d8 WS in its case the first time. Took it out of the case, booted up fine again. Then I put it back in a few weeks later, being extra careful, and all was well.


say, that i made that "mistake". Do i need to took up for a while ( a week ? ), and not to trun it on for that time? ( and took it off from case? )


----------



## DualFiz

Quote:


> Originally Posted by *sagitarius2k*
> 
> say, that i made that "mistake". Do i need to took up for a while ( a week ? ), and not to trun it on for that time? ( and took it off from case? )


I think you misunderstand... what he was saying is that he removed the board from the case and it worked fine (outside of the case). He ran it outside the case for a few weeks before taking the time to put it back into his case.

Basically, if the motherboard is shorting to a standoff, then just removing the standoff from the case or, removing the motherboard from the case is all thats necessary. There's no need to wait to turn it back on again, you can do it the very second you remove the metal causing the short.


----------



## anubis1127

Quote:


> Originally Posted by *DualFiz*
> 
> I think you misunderstand... what he was saying is that he removed the board from the case and it worked fine (outside of the case). He ran it outside the case for a few weeks before taking the time to put it back into his case.
> 
> Basically, if the motherboard is shorting to a standoff, then just removing the standoff from the case or, removing the motherboard from the case is all thats necessary. There's no need to wait to turn it back on again, you can do it the very second you remove the metal causing the short.


Yes, that is what I meant. Sorry for being vague, I was posting from my phone earlier.


----------



## sagitarius2k

Quote:


> Originally Posted by *DualFiz*
> 
> I think you misunderstand... what he was saying is that he removed the board from the case and it worked fine (outside of the case). He ran it outside the case for a few weeks before taking the time to put it back into his case.
> 
> Basically, if the motherboard is shorting to a standoff, then just removing the standoff from the case or, removing the motherboard from the case is all thats necessary. There's no need to wait to turn it back on again, you can do it the very second you remove the metal causing the short.


Quote:


> Originally Posted by *anubis1127*
> 
> Yes, that is what I meant. Sorry for being vague, I was posting from my phone earlier.


Okay, thanks for clarification

I took from my case, and put it on wooden desk. Tried it up, still have problem. LED are up ( start and reset ) but when i press "start" led, no running fan whatsoever is happen.

I also open a help in toms, and heres mine conversation with ASUS reprentatives >> http://www.tomshardware.com/answers/id-1898340/asus-z9pe-problem-working.html

1
hello all

Mine mobo cannot booting to bios. Plug everything to PSU, and when i start it, the fans and PSU not working, but LED is light up. But no q code whatsoever on those.

Is there something wrong? or i forget something to start it up?

thanks

2
Double check all PSU connections again. Check CPU socket to make sure no bent pins. Make sure board is not shorting out against case.

3
Hello mr Raja

yes, i do that exactly you say.

Double check PSU : all 24 and 2x8 are in
CPU bents : try with different xeon
Board against case : first thing to do

anything else ?

4
Clear CMOS, check UEFI and CPU support list to check if your CPU is supported and see if you need UEFI update. If not that, not sure what the problem is.

5
my cpu is xeon e5 2690

can you tell me how to clear CMOS? and check UEFI?

thanks

=========================

So, do i need to do this test?

Thanks


----------



## sagitarius2k

bump

need help pleaseeeeee........


----------



## TheBlademaster01

You have only one CPU in the board? Also what PSU are you using (model)?

You can clear CMOS by pulling out the battery and waiting for 10-20 seconds and then putting it back in.


----------



## beaker7

Did you check for shorts?

If you only have 1 cpu, also ensure it is in CPU socket #1. This is the one that is farther away from the I/O panel.


----------



## sagitarius2k

Quote:


> Originally Posted by *TheBlademaster01*
> 
> You have only one CPU in the board? Also what PSU are you using (model)?
> 
> You can clear CMOS by pulling out the battery and waiting for 10-20 seconds and then putting it back in.


Quote:


> Originally Posted by *beaker7*
> 
> Did you check for shorts?
> 
> If you only have 1 cpu, also ensure it is in CPU socket #1. This is the one that is farther away from the I/O panel.


Mine have :
PSU = corsair axi1200
RAM = teamextreem 2x8 gb x 4 = 64 @ 1600 mhz
CPU = 2 x E5 2690
GPU = GTX 580
case = corsair 900d
CPU fan = noctua NH D14 convert 2011

all except GPU are brand new in box

clear cmos = not try it, will do, thanks
shorts = did you mean when inside case or not? now testing it outside case
1 CPU = not try it, will do, thanks

Oh, i try without cpu or anything ( RAM, HDD, GPU ) just plug into motherboard. Should motherboard display a error code and does PSU also turn on ( fans are spinning ) ? you know, an LED with 00 in it. Because when i strip it, it does not display anything. I mean i blank.

Thanks for answer

edit

what does UEFI mean? asus tech told me to update it. Should I ?


----------



## TheBlademaster01

Quote:


> Originally Posted by *joetesta*
> 
> > First off... Don't ever try to boot a system without a heatsink... You'll burn up a processor really fast that way. Have you been running the processors without them? I sure hope not...
> 
> It has never powered on and ran. The first board we built completely including the coolers, it had the same exact problem (no POST, immediate shutdown) and the heat sinks are a massive pain to put on and take off. They are "COOLER MASTER GeminII S524 120mm Long Life Sleeve CPU Coolers" and 2 of the screws are under the fan and take a decent amount of pressure to attach.
> We sent that first board back 1 week ago and received this one on Wednesday. Before we even put the heat sinks on the second board, we'd figure to make sure it POST's to the bios. But since it never powered on or gave a Q-code, we didn't attach the heat sinks and we closely monitored the CPU for a rise in temperature that never happened at all. We wanted to test each CPU in socket 1 and they both acted the same. If they fried without us even feeling any warmth, i'll eat my socks.
> 
> So based on your experience it does sound as if we need to RMA the board again....
> 
> What's up with that onboard VGA plug and the cable that goes with it; should we expect it to work on boot or does it need drivers?
> 
> thank you,
> Joe


Quote:


> Originally Posted by *joetesta*
> 
> thanks for your help B.T.
> the second 8 pin is connected with this modular cable that plugs into the power supply:
> http://www.newegg.com/Product/Product.aspx?Item=N82E16812162018
> Silverstone Model PP06B-EPS55 21.65" (550mm) Sleeved EPS/ATX12V 8pin Cable


Quote:


> Originally Posted by *joetesta*
> 
> Thanks ZDngr,
> Based on the comments in the Asus support forum, starting April 2012 several users reporting the same issue, we decided to try a different power supply, we're planning to have one of these next week
> http://www.newegg.com/Product/Product.aspx?Item=N82E16817151109
> SeaSonic X-1250 1250W ATX12V / EPS12V SLI Ready 80 PLUS GOLD Certified Full Modular Active PFC Power Supply
> 
> although i'm sure the adapter would be a bit less expensive
> 
> 
> 
> 
> 
> 
> 
> 
> It was before I'd posted here and I think it was already ordered, so we'll see what happens


Quote:


> Originally Posted by *joetesta*
> 
> Great news, we hooked up the new SeaSonic X-1250 1250W power supply and the board boots just fine, we're updating bios now.
> So I don't know whether the cable adapter was incompatible or what was wrong with the other 750W power supply, but we are making progress


Sorry sagitarius, I'm on mobile in the train. Does this help?


----------



## mol075

Hey all, I'm setting up a computer based on the z9pe d8 ws and would appreciate your insight on a few things.

my primary question at the moment regards the windows management software for the on-board RAID controllers. I'm building this system for use in a research lab, but I will not be actively monitoring it, so I would like the task of monitoring it to be simple. The computer has 2 2tb drives in a RAID 1 mirror, without fully considering the consequences I configured the raid array using the LSI controller, assuming ASUS included a non-standard controller for a good reason. From what I've been reading the LSI controller has some issues, and now I'm running into trouble with the MegaRAID Storage Manager for windows. I face-palmed as soon as I saw the UI for the application, I just wanted a simple application to monitor and manage the local raid array, but the application is designed for managing a network of servers, and asks me to log into a server to manage it, despite the fact that the array and application are installed on the same computer. On top of the UI problems, the application won't even load the information once I've logged into the server. It just spins the loading symbol indefinitely.

If someone knows enough about the MegaRAID software to help me get it working (yes I've googled around, can't find any reports matching my description), that'd be great. But at this point I'm thinking the better solution may be to migrate over to the Intel controller and use the RSTe utility. If it is possible to simply use the RSTe utility for managing an array on the LSI controller, that would be the ideal solution, though I don't think that can be done. Barring that, how easily could I transfer the array from the LSI controller to the intel one? I realize this isn't necessarily possible, but with a RAID 1 array I should at least be able to wipe 1 drive, and rebuild the array on the other controller using the complete image on the other drive.

If it's not possible/reasonable to transfer the array to the other controller, I can just re-install everything. It isn't that much of an inconvenience as the computer only has windows and a few applications installed on it so far. But if there is a faster solution I'd like to get this computer out of my office and into the lab.

sorry for the long post, and thanks for your help


----------



## sagitarius2k

Quote:


> Originally Posted by *TheBlademaster01*
> 
> Sorry sagitarius, I'm on mobile in the train. Does this help?


Thanks you blademaster. That very exactly what i have been trough. So the problem was cable that did not compatible? holy poop.

So, based on your suggestion, this one can do that > http://www.newegg.com/Product/Product.aspx?Item=N82E16812162018

one more thing, did he RMA that board, again?

Thanks you blademaster, will try soon, after i find that, thing


----------



## TheBlademaster01

Someone may need to confirm. I'm not sure what the issue really is since I never had this happen to me. I just remembered this and thought I'd post this


----------



## sagitarius2k

okay, the problem now has been solved. Yup, it's psu. I am using corsair 900d and corsair axi1200. the problem is, the cable of 24 pin and 8 X 2 pin are too short to plug into motherboard. It cause the cable to bend too much, it cause psu not working.

Now just wait for cable extension









thanks all. Asus should link this thread to become official thread for installation, maintaining, and troubleshooting


----------



## TheBlademaster01

Ah, nice you found the problem. Troubleshooting can be so annoying with these boards


----------



## anubis1127

Very nice, glad you got it sorted!


----------



## joetesta

Quote:


> Originally Posted by *sagitarius2k*
> 
> one more thing, did he RMA that board, again?


No we didn't - with the new Power Supply it's been running solid








Glad you got yours working as well


----------



## tanner2

Hi everyone, glad to have found all your info here.
I have a question. Been looking for chips and have run into L1 and L2 stepping on the e5 26xx.
Does the bios support that? If so which one?
My board is a asus z9pe-d16/2l getting it Monday.
Just will use it to fold and crunch.


----------



## sweesecheese

Hi,
Just been building a new PC on a Z9PE-D8 WS board with 2xE5-2650v2's.

It was working nicely up to the point where I put in a Tesla C2075.
Windows 8.1 restarted successfully and saw, but didn't recognise the card.
I installed the NVidia drivers and rebooted.
Then Windows wouldn't start and showed a blue screen (not of death, just blank and blue).

I restarted again and now Bios won't even start.
The Q-codes show 19 repeatedly (pre-memory PCH installation is started) and then finally shows bF (unknown).

I am guessing that something has failed on the board and it needs to be sent back, but am reluctant to dismantle everything before I have tried all alternatives.

Any suggestions for what I might try?

Many thanks


----------



## TheBlademaster01

Try putting the GPU in different slots.


----------



## sweesecheese

Quote:


> Originally Posted by *TheBlademaster01*
> 
> Try putting the GPU in different slots.


Thanks for the suggestion, but I should have said, when BIOS didn't start, I took out everything non-essential.
It is now down to power, CPUs, RAM and VGA plug and still no luck.


----------



## DualFiz

Quote:


> Originally Posted by *sweesecheese*
> 
> Thanks for the suggestion, but I should have said, when BIOS didn't start, I took out everything non-essential.
> It is now down to power, CPUs, RAM and VGA plug and still no luck.


Have you reset the bios?

It's possible a setting has become corrupt.

Also, I believe there's a jump on the motherboard you need to enable for correct functionality of the Tesla card. I think it's between the 3rd and 4th PCIE slot. Look in the motherboard.

But yeah, start by resetting the bios. (via the jumper)


----------



## sweesecheese

Quote:


> Originally Posted by *DualFiz*
> 
> Have you reset the bios?
> 
> It's possible a setting has become corrupt.
> 
> Also, I believe there's a jump on the motherboard you need to enable for correct functionality of the Tesla card. I think it's between the 3rd and 4th PCIE slot. Look in the motherboard.
> 
> But yeah, start by resetting the bios. (via the jumper)


Thanks, I hadn't tried that, but now I did. Still no luck.
Also thanks for the jumper tip. I hope someday to get back to a point when I can use it.


----------



## DualFiz

Quote:


> Originally Posted by *sweesecheese*
> 
> Thanks, I hadn't tried that, but now I did. Still no luck.
> Also thanks for the jumper tip. I hope someday to get back to a point when I can use it.


No problem. I'm sorry the idea didn't solve the problem. I've not much experience with this motherboard as of yet, but I've had other motherboard problems resolved by clearing the bios.

Do you have another power supply you could test with? I wonder if the draw from the Tesla card was too much and damaged the PSU somehow?


----------



## sweesecheese

Quote:


> Originally Posted by *DualFiz*
> 
> Do you have another power supply you could test with? I wonder if the draw from the Tesla card was too much and damaged the PSU somehow?


Thanks for your efforts. Resetting the BIOS has reinforced my theory that something failed on the MB.
I have now given up and returned the MB to the retailer for replacement.

I don't have another PSU and hope that it isn't damaged, but this is a 1500W PSU and I never saw it running above 250W, so it should have been able to handle an additional draw (up to 225W) from a healthy Tesla. If the Tesla is unhealthy, who knows, but I never got far enough to give it any computing load.

I had configured the PSU as single rail, as I planned eventually to add a number of Xeon Phis and I wondered whether that could have overwhelmed the MB somehow.


----------



## DualFiz

Quote:


> Originally Posted by *sweesecheese*
> 
> Thanks for your efforts. Resetting the BIOS has reinforced my theory that something failed on the MB.
> I have now given up and returned the MB to the retailer for replacement.
> 
> I don't have another PSU and hope that it isn't damaged, but this is a 1500W PSU and I never saw it running above 250W, so it should have been able to handle an additional draw (up to 225W) from a healthy Tesla. If the Tesla is unhealthy, who knows, but I never got far enough to give it any computing load.
> 
> I had configured the PSU as single rail, as I planned eventually to add a number of Xeon Phis and I wondered whether that could have overwhelmed the MB somehow.


Are you using the EVGA Classified PSU? Just curious.

Regardless, I am sorry to hear of yet another problematic experience with this motherboard. Who knows what happened, but what we do know is that it shouldn't have happened at all.

Obviously as I've just purchased this board for a build myself, I find myself somewhat concerned of it's long term stability.

Best wishes with the RMA process, hopefully you do not suffer much down time.


----------



## sweesecheese

Quote:


> Originally Posted by *DualFiz*
> 
> Are you using the EVGA Classified PSU? Just curious.
> 
> Regardless, I am sorry to hear of yet another problematic experience with this motherboard. Who knows what happened, but what we do know is that it shouldn't have happened at all.
> 
> Obviously as I've just purchased this board for a build myself, I find myself somewhat concerned of it's long term stability.
> 
> Best wishes with the RMA process, hopefully you do not suffer much down time.


Thanks for your kind regards.

I doubt my experience over 2 days of use says much about the long-term stability of the board.
Certainly hope not!

Yes, it is the EVGA Classified PSU. Quite impressed with how clearly they explained and laid everything out, build quality etc.


----------



## DualFiz

I have a question for owners of this board... regarding CPU temps.

Currently, I am only at the start of testing a new build using this motherboard and two 8-core Xeon E5-V2 2.8ghz (turbo at 3.2ghz) 130watt CPU's. Cooling them (for now) are two Dynatron R15 passive copper heatsinks. Obviously the case these procs are running in has a cooling system in place, to compensate.

With that, when sitting in the BIOS (so, CPU's should be at "load" and "max" speed), the temps being reported are right around the 80c mark.

I am new to this generation of Intel processor, so I don't know if thats a fair temp or not, but it feels far too warm for me.

I'm not familiar with ASUS, or how their BIOS reports CPU temps, so I don't know if things would be lower while running in an operating system (windows/linux), or if the cooling system simply isn't enough to manage these beefy processors.

Any input on YOUR current temps would be appreciated.

Thanks!


----------



## ZDngrfld

The highest core temp I'm seeing is 45° C on my E5-2650s at full load. I'm going to be replacing them with V2s shortly.


----------



## DualFiz

Quote:


> Originally Posted by *ZDngrfld*
> 
> The highest core temp I'm seeing is 45° C on my E5-2650s at full load. I'm going to be replacing them with V2s shortly.


Wow! Thats a pretty incredible load temp... I assume these are watercooled? As like those in your system configuration links?

Also, I will be very interested to see how your temps are after upgrading to the v2 Xeons. From everything I've read, they run HOT.


----------



## ZDngrfld

Quote:


> Originally Posted by *DualFiz*
> 
> Wow! Thats a pretty incredible load temp... I assume these are watercooled? As like those in your system configuration links?
> 
> Also, I will be very interested to see how your temps are after upgrading to the v2 Xeons. From everything I've read, they run HOT.


Nope, not watercooled. Just a couple Hyper-212 Evos. My only watercooled system anymore is my SR-2. I'm assuming they run hot from the track record of desktop IVB chips.


----------



## DualFiz

Quote:


> Originally Posted by *ZDngrfld*
> 
> Nope, not watercooled. Just a couple Hyper-212 Evos. My only watercooled system anymore is my SR-2. I'm assuming they run hot from the track record of desktop IVB chips.












Extra impressive then. I actually toyed with the idea of picking up a set of Hypers for this system, but ultimately I believe it is going to get moved from the tower to a 2u or 3u rack. Still, for now it might not be a bad idea to pop in a set of larger coolers and see what sort of temperatures I get.

The real problem is I can't find any real conclusive information regarding the max thermal range of the new v2 Xeons. Intel might say one thing, but others will say something different. Of course I suppose Intels opinion is the only one that should matter.

So, as long as it's under, say 85c, I guess I shouldn't worry.


----------



## ZDngrfld

I'll be cooling my V2s with the Hyper-212s so it'll be a good comparison. I know there's some guys over at [H] that are running the V2s. Maybe they have temps posted so you can compare


----------



## DualFiz

One more question for those in this thread... who might know.

I'm currently using a set up ES v2 CPU's, with BIOS 5103, which is what it shipped with. Is there any risk in flashing the latest BIOS 5304? I know in the past certain ES CPU's have lost support through BIOS updates, but do not know if this holds true for V2 Xeons or not.

Thanks again!


----------



## TubbeAh

Hi,

I'm a bit curious to what you all use for CPU cooling on this board. I'm currently running two Dynatron r17, but the performance so far is just ok. My temp is around 68-70 C on full load (e5-2670). Checked with ASUS what their recommendations were and got this interesting answer regarding weight: "650 grams and beyond can cause warp damage and cause direct damage, so we dont suggest anything beyond this."

So, the big coolers are no no then?? I can certainly see a problem with 2+ kg on a long term...

The Noctua NH-U9DX i4 caught my eye as a pretty light weight and silent cooler (just slightly under 650g). Do you think the performance of this one is better than the dynatron? Hard to tell from figures...


----------



## sweesecheese

Quote:


> Originally Posted by *DualFiz*
> 
> One more question for those in this thread... who might know.
> 
> I'm currently using a set up ES v2 CPU's, with BIOS 5103, which is what it shipped with. Is there any risk in flashing the latest BIOS 5304? I know in the past certain ES CPU's have lost support through BIOS updates, but do not know if this holds true for V2 Xeons or not.
> 
> Thanks again!


5304 worked for me with dual E5-2650 v2s
The pre-installed BIOS only worked with one.


----------



## ZDngrfld

Quote:


> Originally Posted by *TubbeAh*
> 
> Hi,
> 
> I'm a bit curious to what you all use for CPU cooling on this board. I'm currently running two Dynatron r17, but the performance so far is just ok. My temp is around 68-70 C on full load (e5-2670). Checked with ASUS what their recommendations were and got this interesting answer regarding weight: "650 grams and beyond can cause warp damage and cause direct damage, so we dont suggest anything beyond this."
> 
> So, the big coolers are no no then?? I can certainly see a problem with 2+ kg on a long term...
> 
> The Noctua NH-U9DX i4 caught my eye as a pretty light weight and silent cooler (just slightly under 650g). Do you think the performance of this one is better than the dynatron? Hard to tell from figures...


I'm running Hyper-212s on almost all my multi-socket rigs (Asus Z9PE-D8 WS, Asus Z9NA-D6C, Supermicro X9DRL-3F, Supermicro H8QGi-+F) with no problems. They weigh in at 580 grams and keep them all very cool. You'll just need a big enough case for them to fit.


----------



## DualFiz

Quote:


> Originally Posted by *sweesecheese*
> 
> 5304 worked for me with dual E5-2650 v2s
> The pre-installed BIOS only worked with one.


Thank you for the info!

...were those CPU's "ES" models? Or production?

Thanks again.


----------



## sagitarius2k

Quote:


> Originally Posted by *TubbeAh*
> 
> Hi,
> 
> I'm a bit curious to what you all use for CPU cooling on this board. I'm currently running two Dynatron r17, but the performance so far is just ok. My temp is around 68-70 C on full load (e5-2670). Checked with ASUS what their recommendations were and got this interesting answer regarding weight: "650 grams and beyond can cause warp damage and cause direct damage, so we dont suggest anything beyond this."
> 
> So, the big coolers are no no then?? I can certainly see a problem with 2+ kg on a long term...
> 
> The Noctua NH-U9DX i4 caught my eye as a pretty light weight and silent cooler (just slightly under 650g). Do you think the performance of this one is better than the dynatron? Hard to tell from figures...


Quote:


> Originally Posted by *ZDngrfld*
> 
> I'm running Hyper-212s on almost all my multi-socket rigs (Asus Z9PE-D8 WS, Asus Z9NA-D6C, Supermicro X9DRL-3F, Supermicro H8QGi-+F) with no problems. They weigh in at 580 grams and keep them all very cool. You'll just need a big enough case for them to fit.


Where did that "650 grams and beyond can cause warp damage and cause direct damage, so we dont suggest anything beyond this." came from? you link for that?

Now i am using noctua NH D14 in 900D.


----------



## anubis1127

Quote:


> Originally Posted by *ZDngrfld*
> 
> Nope, not watercooled. Just a couple Hyper-212 Evos. My only watercooled system anymore is my SR-2. I'm assuming they run hot from the track record of desktop IVB chips.


The v2 Xeons do not have crappy TIM like the desktop Ivy chips.


----------



## TubbeAh

Quote:


> Originally Posted by *sagitarius2k*
> 
> Where did that "650 grams and beyond can cause warp damage and cause direct damage, so we dont suggest anything beyond this." came from? you link for that?
> 
> Now i am using noctua NH D14 in 900D.


It is a quote from one of the ASUS support tech people I had mail contact with.


----------



## TheBlademaster01

Quote:


> Originally Posted by *DualFiz*
> 
> Thank you for the info!
> 
> ...were those CPU's "ES" models? Or production?
> 
> Thanks again.


I think it rather had to do with the Isoc setting in the BIOS. Having it on caused IVB-EP CPUs to fail in dual mode. Disabling it solved the problem according to psyq. 5004 had it on by default while the latest release disabled it by default

Quote:


> Originally Posted by *anubis1127*
> 
> The v2 Xeons do not have crappy TIM like the desktop Ivy chips.


This. Coupled with lower power consumption it should probably have somewhat lower temps. Maybe not so much because of higher transistor density and smaller surface area.

I'm using 2x EK Supremacy on 360 rad to cool the 2665s @2.5. Max temps are 38*C with ambients of about 15*C.


----------



## sweesecheese

Quote:


> Originally Posted by *DualFiz*
> 
> ...were those CPU's "ES" models? Or production?


Just regular shop-bought, so presumably production.


----------



## NapalmV5

guys how are titans working on this mobo and ivys? any issues? pcie3 issues? games/apps/benchmarks issues?

anyone got tomb raider/witcher 2/just cause 2/portal 2 working with 2 cpus ?

112 bus


111 bus


----------



## ZDngrfld

Quote:


> Originally Posted by *NapalmV5*
> 
> guys how are titans working on this mobo and ivys? any issues? pcie3 issues? games/apps/benchmarks issues?
> 
> anyone got tomb raider/witcher 2/just cause 2/portal 2 working with 2 cpus ?
> 
> 112 bus
> 
> 
> Spoiler: Warning: Spoiler!


So you're able to go over the ~104 BCLK cap that most everyone was hitting with SB-E? I'm liking this...


----------



## TheBlademaster01

Yeah, I've seen someone do that as well. He was in this thread with E5-2696v2. I wonder what we're doing wrong or they are doing right.


----------



## DualFiz

Hmm... I wonder if the V2 Xeons just handle higher FSB differently than the V1's do?


----------



## Donkey1514

Is that 1 chip or 2 chips? IIRC, @anubis1127 did a higher bclk with 1 chip vs 2 chips


----------



## TheBlademaster01

That has to be a dual CPU score. I get ~1700 pts with my system. It also says 16/32 (bad Intel for crippling 2687WV2







)


----------



## Donkey1514

Quote:


> Originally Posted by *TheBlademaster01*
> 
> That has to be a dual CPU score. I get ~1700 pts with my system. It also says 16/32 (bad Intel for crippling 2687WV2
> 
> 
> 
> 
> 
> 
> 
> )


Didn't see that









i'll run Cinebench R15 tonight on my 2687 v1's tonight for a comparison


----------



## NapalmV5

2 cpus
Quote:


> Originally Posted by *ZDngrfld*
> 
> So you're able to go over the ~104 BCLK cap that most everyone was hitting with SB-E? I'm liking this...


102-103 max @ radeon 290 (pcie3)
~114 windows boot @ gtx 580 (pcie2)

gpu plays a big role how high the bus goes
Quote:


> Originally Posted by *NapalmV5*
> 
> guys how are titans working on this mobo and ivys? any issues? pcie3 issues? games/apps/benchmarks issues?
> 
> anyone got tomb raider/witcher 2/just cause 2/portal 2 working with 2 cpus ?


anyone?


----------



## TheBlademaster01

bios rev 5004 and later on E5 v2s work correctly with PCI-e 3.0 and Titans. AndyE and psyq run them AFAIK.

And yes, that makes sense. Some GPUs have issues at higher than 104MHz bus speed. Thanks for that piece of information man


----------



## Donkey1514

Quote:


> Originally Posted by *NapalmV5*
> 
> 2 cpus
> 
> ~114 windows boot @ gtx 580 (pcie2)


I need to buy a gtx580


----------



## NapalmV5

Quote:


> Originally Posted by *TheBlademaster01*
> 
> bios rev 5004 and later on E5 v2s work correctly with PCI-e 3.0 and Titans. AndyE and psyq run them AFAIK.
> 
> And yes, that makes sense. Some GPUs have issues at higher than 104MHz bus speed. Thanks for that piece of information man


and thanks for that piece of information too man


----------



## TheBlademaster01

I know my old GTX 285 and 9800GTX could run 110MHz plus but I hadn't tried my GTX 470 yet. Apparently it doesn't.

Don't you have any more GPUs to test Donkey?


----------



## DualFiz

Quote:


> Originally Posted by *TheBlademaster01*
> 
> I think it rather had to do with the Isoc setting in the BIOS. Having it on caused IVB-EP CPUs to fail in dual mode. Disabling it solved the problem according to psyq. 5004 had it on by default while the latest release disabled it by default


Excellent, thank you for the information!

I assume then the latest bios would be alright to flash even with ES CPU's. But, I suppose, nobody ever really knows for sure.

Thanks again.


----------



## TheBlademaster01

Here a confirmation

Quote:



> Originally Posted by *psyq3212*
> 
> JFYI, Z9PE D8 WS BIOS 5103 contains production microcode update for IvyTown CPUs. Yay
> 
> 
> 
> 
> 
> 
> 
> 
> 
> Another thing, for 12 core Ivy Bridge EP CPUs, you should disable Isoc in the QPI configuration.
> 
> My system is now fully stable, even with NVIDIA Titan running @PCIe 3.0 link speed.


----------



## NapalmV5

Quote:


> Originally Posted by *Donkey1514*
> 
> I need to buy a gtx580


have you guys tried pcie1 gpus ?


----------



## ZDngrfld

What about the onboard video? Maybe that'll work? I don't have the thing to plug it into the header to test it or else I would.


----------



## DualFiz

Quote:


> Originally Posted by *TheBlademaster01*


Hmm... very interesting. Thanks again!

I wonder what that setting does anyways. There are so many QPI settings that don't seem to have any real measurable impact on performance.


----------



## TheBlademaster01

I figure I should just quote my sources since I haven't really tried any of this lol

Quote:



> Originally Posted by *psyq3212*
> 
> *Isoc is a mode of transmission between two QPI nodes (Isochronous traffic).* It should be ON for 2P configurations, but it should be disabled for 12 core IvyTown 2P configurations. For CPUs with 10 cores and less, it should be ON in 2P configurations.
> 
> For HEDT platforms, it only make sense for the old Nehalem/Westmere CPUs which used QPI for connection with the IOH.
> 
> SNB-E and IVB-E HEDT parts do not use QPI links at all, as QPI is only used for connection between multiple CPUs and there is only one CPU, so it does not really matter for those parts.
> 
> I could not pair TITAN on my old SNB-EP @ PCIe 3.0 as I had B0 and C0 steppings in the past.
> 
> --
> 
> As for B0 SNB-EP I do not have those anymore so unfortunately I cannot test, but I am almost 100% sure it does not work since there were too many MRC changes for ASUS to bother to restore compatibility
> 
> 
> 
> 
> 
> 
> 
> 
> 
> @edit - I made a small mistake, Isoch should be OFF for 12-core parts only.


----------



## TheBlademaster01

Also, I have a 7300LE PCI-e 1.0. If the board had a PCI slot that would have been epic (already popular for benchers).

E:

It doesn't have a heatsink...


----------



## ZDngrfld

I'm using a Quadro NVS 285 on mine. It's PCI-e 1.0... I couldn't get over a 104 BCLK.


----------



## Donkey1514

Quote:


> Originally Posted by *TheBlademaster01*
> 
> I know my old GTX 285 and 9800GTX could run 110MHz plus but I hadn't tried my GTX 470 yet. Apparently it doesn't.
> 
> Don't you have any more GPUs to test Donkey?


I have a gtx 780, gtx 550ti, and a hd6450. I'll try each tonight and report back.

@NapalmV5- Did you mess with any voltages?


----------



## DualFiz

Quote:


> Originally Posted by *TheBlademaster01*


Again, very interesting!

Thanks!

It appears the CPU's I'm currently testing are B0 and C1, so, I would hazard a guess that they're not going to work correctly with that function enabled. It boots fine with everything at default, but I've yet to install an OS and actually begin true compatibility testing.

We will see.

Thanks again!


----------



## ignoramus

Can someone here do me a favour?

I cant run photosynth on this machine (2 x 2665s, 104bclk, GTX460) and i dont know why!

Can someone d/l it quick and tell me if you can run it on your 2p machine? i suspect i have bad ram considering how cheap it was but it might just be having issues with the 2p setup. Thanks in adavance!

//I realise this might not be that simple.. you have to reg an account and actually chose photos to synth together so maybe if someone that already has it installed could let me know if it works. Thanks.


----------



## TheBlademaster01

I would if I were in Windows, sorry


----------



## NapalmV5

*and heres that 114 bus*


----------



## TheBlademaster01

That's insane









What did you change to get that stable?


----------



## NapalmV5

i kept "force ddr1866" just like at 111 bus instead of lowering to "force 1600" timings too low and wasnt stable.. running like champ now


----------



## TheBlademaster01

No voltage changed?


----------



## NapalmV5

no same voltage.. i changed oses but same limits same results


----------



## TheBlademaster01

I seriously can't post above 104MHz no matter what =/


----------



## NapalmV5

try different gpus at least try a 580 if you dont get any higher bus than its the cpus if ram is not in the way


----------



## sagitarius2k

Quote:


> Originally Posted by *NapalmV5*
> 
> *and heres that 114 bus*


hello napalmv5

I have :
e5 2690
teamextreem 2133 8x8 = 64 gb
Gtx 580

what do you recommended for this?


----------



## NapalmV5

Quote:


> Originally Posted by *sagitarius2k*
> 
> hello napalmv5
> 
> I have :
> e5 2690
> teamextreem 2133 8x8 = 64 gb
> Gtx 580
> 
> what do you recommended for this?


leave ram timings to auto set "force ddr 1866" and increase bus one by one


----------



## Jann

I know it's a long shot, but has anyone here installed OSX on this motherboard?
I'm on 5004 bios, and couldn't get it to work when I tried a while back. But still would like to do that one day









As for cooling, in a Xigmatek Elysium with default cooling my two E5-2660 C0 cpus never go over 48-50 celsius using 2 CM Hyper 212 Evo coolers. Ambient about 20-25c.
Even during last summer in a different room with bad heat it never got over 60 under full load


----------



## NapalmV5

114 bus + 580 sli



i went back to the first z9 and got the same limits same results

2x z9
2x 2687w v2
2x 4x4gb crucial blp 1600
2x true copper
2x gtx 580

114 bus max


----------



## TheBlademaster01

Lol, I didn't even know that true fits on S2011









That's just epic though. I might just need to get a different GPU lol.


----------



## sweesecheese

Hi folks,
back after replacing a dud motherboard a few pages ago, but still struggling.

Windows 8.1 keeps restarting or freezing with an error in system logs Kernel Power event code 41.

Everyone tells me this is memory-related.
I am using 8 x 8GB of Kingston HyperX 1866 non-ECC running at 1600MHz.

I disabled Legacy USB in BIOS because I read that it caused memory conflicts.

memcheck86+ ran twice (so far) successfully, but Windows keeps failing.

Any ideas what might cause this?
Has anyone tried this memory or equivalent on this board?
It isn't on the Asus approved list, but not a lot is.

Thanks

Other details:
- 2 Xeon E5-2650v2
- NVidia Quadro NVS295
- EVGA NEX 1500 Classified


----------



## Krobar

Quote:


> Originally Posted by *sagitarius2k*
> 
> wow, is that titan really hunger for power? i planed to use mine rig with r9 290x CF or GTX 780i SLI. Should 1200 watt corsair 1200AXi is enough for this?


I would have thought that is excessive. My Brother has a Supermicro board and runs Sli 680s, dual E5-4617s, 6 mechanical Drives, hardware raid and 24 Registered RAM sticks and still only sees around 650-700W peak load when system is under heavy synthetic load (Furmark, Linpack, sequential disk and memory tests simultaneously). That system runs fine with an 860W corsair.


----------



## anubis1127

780Tis us a significant bit more power than 680s though. Some people also like to stay within the PSUs maximum efficiency rating, which typically is not near maxed out.


----------



## TheBlademaster01

Yeah, peak efficiency is at 50% load.


----------



## anubis1127

Well it depends on the PSU, but yeah, its generally between 50-60%. I usually shoot for around 50%.


----------



## TheBlademaster01

Well, there's actually a theorem that the 50% peak is based on (thevenin). It should be the same for all PSUs though the rolloff should be different. They also are more efficient at 230V50Hz vs 115V60Hz.


----------



## Krobar

Quote:


> Originally Posted by *TheBlademaster01*
> 
> Well, there's actually a theorem that the 50% peak is based on (thevenin). It should be the same for all PSUs though the rolloff should be different. They also are more efficient at 230V50Hz vs 115V60Hz.


System mentioned earlier was not that efficient at idle (Mid 80s) but >90% under load; I usually shoot for about 85% of peak after sensible upgrades (eg.SLI if only on single card currently) because otherwise idle efficiency can be terrible.


----------



## LeMakisar

Quote:


> Originally Posted by *NapalmV5*
> 
> guys how are titans working on this mobo and ivys? any issues? pcie3 issues? games/apps/benchmarks issues?
> 
> anyone got tomb raider/witcher 2/just cause 2/portal 2 working with 2 cpus ?


Exact same problem with Portal 2, I've spent a lot of time with Valve support but it was useless ... luckily I still have another computer so I could finish this wonderful game









Deus Ex : HR also has issues but you just have to restrict the steam.exe to a few cores and it just works.

GRID doesn't work either (a shame :/)

If someone has information to make these work that would be nice indeed.


----------



## LeMakisar

By the way, I have a question : I tried that more than a year ago but without luck : is it possible with the latest bios to force DDR to 1866 with 2 2687W (Sandy) ? Last time I tried, the mobo woudln't post, but maybe it has changed.

And while I'm on the forum ... is there any replacement for this mobo coming ? are the future haswell-e mobos going to support sandy bridge e ?
I'd like to pick a replacement card before they all disappear









thanks !


----------



## anubis1127

I can't answer many of your questions, but haswell-ep is going to be a new socket.


----------



## TheBlademaster01

I haven't heard of sandy being able to do 1866 yet

Hsw-e will not be compatible with sandy e and ivy e. It will have different keying (2011-3 vs 2011-0)


----------



## Krobar

Quote:


> Originally Posted by *TheBlademaster01*
> 
> I haven't heard of sandy being able to do 1866 yet
> 
> Hsw-e will not be compatible with sandy e and ivy e. It will have different keying (2011-3 vs 2011-0)


I could get post on the SM board at 1866 with 4 1333 reg sticks per Sandy but it would crash after a couple of minutes. My guess is that any limitation is probably a board/bios issue rather than a CPU issue.

Haswell-E looks like a true next gen product and I would love to get at least a single cpu setup on release; hopefully the lack of competition will not boost pricing too significantly.


----------



## Donkey1514

Quote:


> Originally Posted by *Krobar*
> 
> I could get post on the SM board at 1866 with 4 1333 reg sticks per Sandy but it would crash after a couple of minutes. My guess is that any limitation is probably a board/bios issue rather than a CPU issue.
> 
> Haswell-E looks like a true next gen product and I would love to get at least a single cpu setup on release; hopefully the lack of competition will not boost pricing too significantly.


I know someone that has a ES Haswell-EP chip with 14c/28t if you're interested in purchasing


----------



## lowfat

Quote:


> Originally Posted by *Donkey1514*
> 
> I know someone that has a ES Haswell-EP chip with 14c/28t if you're interested in purchasing


What is point if there are no boards available for it?


----------



## Donkey1514

Quote:


> Originally Posted by *lowfat*
> 
> What is point if there are no boards available for it?


you hush!


----------



## pringleman

Greetings guys,

I soon to follow suit with this motherboard. I would really like not have air cooling if it's not going to be a great effort. From my research and having no experience in water cooling leads me to believe this to be overly hard. Can any one point me to info relating to dual water cooling setups for this motherboard with 2 xeon cpu's. Thanks you


----------



## ZDngrfld

Quote:


> Originally Posted by *pringleman*
> 
> Greetings guys,
> 
> I soon to follow suit with this motherboard. I would really like not have air cooling if it's not going to be a great effort. From my research and having no experience in water cooling leads me to believe this to be overly hard. Can any one point me to info relating to dual water cooling setups for this motherboard with 2 xeon cpu's. Thanks you


You could run two closed loop setups. Something like two Corsair H80s


----------



## beaker7

Quote:


> Originally Posted by *ZDngrfld*
> 
> You could run two closed loop setups. Something like two Corsair H80s


I am getting ready to upgrade to E5-2687wv2's and dual H90's.


----------



## 364958

I'm looking at purchasing this board for my upcoming Hackintosh build, to replace my Windows-based (now old) SR-2 build.

I have a few questions,

A. Does v2 Xeons work with this board?

B. Have you had any difficulty with the boards? (i.e. DOAs/RMAs, faults, or problems getting it to work correctly)

C. Would you purchase this board again?

D. I can't find any current competition to this board, is this the only high-end enthusiast dual LGA 2011 board available?

E. What case/enclosure are you using to currently hold the motherboard?


----------



## ZDngrfld

Quote:


> Originally Posted by *Kree*
> 
> I'm looking at purchasing this board for my upcoming Hackintosh build, to replace my Windows-based (now old) SR-2 build.
> 
> I have a few questions,
> 
> A. Does v2 Xeons work with this board?
> B. Have you had any difficulty with the boards? (i.e. DOAs/RMAs, faults, or problems getting it to work correctly)
> C. Would you purchase this board again?
> D. I can't find any current competition to this board, is this the only high-end enthusiast dual LGA 2011 board available?
> E. What case/enclosure are you using to currently hold the motherboard?



Yes. You need to be running at least bios version 5206. So you need to make sure that the board you buy has that version or you'll need an older CPU to flash. You can also buy a bios chip with the most current version.
This thread is pretty much here because of the RMAs and issues people have with this board. I'd suggest reading through at least some of the more current pages
No
I hope you're not expecting to overclock much, because you can't. You'll get a max BCLK of 114 it's looking like. That hinges upon other components allowing that to happen. Supermicro has their X9DAX boards.
I have mine in an old Coolermaster Cosmos-S


----------



## 364958

Would you consider this board an upgrade from the EVGA SR2, back from the X58 days?


----------



## ZDngrfld

Quote:


> Originally Posted by *Kree*
> 
> Would you consider this board an upgrade from the EVGA SR2, back from the X58 days?


I'm still using my SR-2 as my gaming rig. I use the Z9 for folding/mining. The E5s definitely win out in performance and wattage. Maxed out my SR-2 runs at ~600 watts, the Z9 with two E5 V2s in it hit 345 watts. If I threw a video card in it I bet it'll still be under 500 watts fully pegged.


----------



## anubis1127

Quote:


> Originally Posted by *Kree*
> 
> I'm looking at purchasing this board for my upcoming Hackintosh build, to replace my Windows-based (now old) SR-2 build.
> 
> I have a few questions,
> 
> A. Does v2 Xeons work with this board?
> 
> B. Have you had any difficulty with the boards? (i.e. DOAs/RMAs, faults, or problems getting it to work correctly)
> 
> C. Would you purchase this board again?
> 
> D. I can't find any current competition to this board, is this the only high-end enthusiast dual LGA 2011 board available?
> 
> E. What case/enclosure are you using to currently hold the motherboard?


My answers don't vary much from @ZDngrfld's, with the exception of C. Yes, and E. Fractal Define XL R2


----------



## TheBlademaster01

Quote:


> Originally Posted by *Kree*
> 
> I'm looking at purchasing this board for my upcoming Hackintosh build, to replace my Windows-based (now old) SR-2 build.
> 
> I have a few questions,
> 
> A. Does v2 Xeons work with this board?
> 
> B. Have you had any difficulty with the boards? (i.e. DOAs/RMAs, faults, or problems getting it to work correctly)
> 
> C. Would you purchase this board again?
> 
> D. I can't find any current competition to this board, is this the only high-end enthusiast dual LGA 2011 board available?
> 
> E. What case/enclosure are you using to currently hold the motherboard?


A. Yes, from bios ver. 5004 on (ver. 5206 and later is recommended due to bug fixes and DDR non-registered/ECC support)

B. No I got mine used from an HPC cluster somewhere in Westfalen (GER), it was used in combination with Tesla boards without issues over ~7 months

C. Yes, used

D. Used/old-stock SR-X if you are lucky. Otherwise like ZDngr said, you need to look at Supermicro's offerings though you lose RAM and CPU configuration settings (timings and bClk OC)

E. SST Raven RV01B-W (check the link "Pernod" in my sig for a build log)


----------



## pringleman

Quote:


> Originally Posted by *ZDngrfld*
> 
> You could run two closed loop setups. Something like two Corsair H80s


Quote:


> Originally Posted by *beaker7*
> 
> I am getting ready to upgrade to E5-2687wv2's and dual H90's.


Thanks, some advice lingering about on the web was that watercooling kits were no good but they're probably are good enough.


----------



## pringleman

I have seen these ES samples on ebay.

Could anyone advise me on whether it is worth buying these.

On the start of the thread there is a processor compatibility chart and under the b1 title it reports "no users feedback" Can't find any more info on this and it looks to be a bit of gamble even though engineering samples have worked for people in the past. Many thanks.


----------



## lowfat

Quote:


> Originally Posted by *Kree*
> 
> Would you consider this board an upgrade from the EVGA SR2, back from the X58 days?


Depends on what you'd be using for CPUs. I'd personally say no unless you can afford some of the ridiculously expensive higher clocked E5s.

I definitely wouldn't buy the Z9PE-D8 WS again.


----------



## ZDngrfld

Quote:


> Originally Posted by *TheBlademaster01*
> 
> D. Used/old-stock SR-X if you are lucky. Otherwise like ZDngr said, you need to look at Supermicro's offerings though you lose RAM and CPU configuration settings (timings and bClk OC)


You can overclock Supermicro's X9DAX boards. That's their "Hyper-Speed" feature


----------



## TheBlademaster01

Yeah, I know. There are multiple SM boards that have hyper speed, but none that have timings setting and such extensive settings. I think Sr-X and Z9 are unique in that.


----------



## TheBlademaster01

Aside from USB, PCI-E and SLI/CF compatibility.


----------



## gurmukh23

Hi Guys,

I am building the rig below

PSU: EVGA Supernova 1300W
Case: Corsair Obsidian Series 900D
Motherboard: ASUS Z9PE-D8 WS Dual LGA 2011
CPU: 2 x (Intel Xeon E5-2697 v2 12 core)
Memory: CORSAIR Dominator Platinum 64GB (8 x 8GB) or 2 x (Kingstons 64GB ECC Kit)
GPU: 3 x (EVGA GTX Titan) Not SLI
SSD: SAMSUNG 840 Pro Series 256GB
HDDs: 3 x Western Digital WD VelociRaptor 1TB
CPU Cooler: 2 x (Xigmatek Dark Knight)
Others: Blu Ray Writer, Card Reader, Case Fans

My concern is the PSU, will 1300w be enough? WIll air cooling be ok?

Really appreciate the help, am building this after 5 years so am very new to high end rigs


----------



## 364958

I've decided that I'm going to go for the Asus board.

Here is my current configuration:

*CPU*: 2x Intel Xeon E5-2697 V2

*GPU*: 4x AMD Radeon R9 290X's (XFX, Sapphire, MSI & PowerColor)

*RAM*: 64GB of Hynix DDR3-1600, ECC, CAS = 11-11-11

*PSU*: Enermax EPM1500EGT (1.5Kw)

*SSD*: Samsung 840 EVO.

*MoBo*: Asus Z9PE-D8

*Cooling*: 2x Zalman Reserator 3's.

*Case*: Silverstone Raven RV03B.

Quote:


> Originally Posted by *lowfat*
> 
> Depends on what you'd be using for CPUs. I'd personally say no unless you can afford some of the ridiculously expensive higher clocked E5s.
> 
> I definitely wouldn't buy the Z9PE-D8 WS again.


*The computer's primary tasks would be*:

- Running 3x 4K Monitors

- Encrypting & Hashing (AES256/SHA256/RSA4098/Salsa20)

- Gaming (Crysis, ArmA 3)

- Photo Editing

- Programming

- Hosting TS3/Mumble Server

- Hosting Game Servers (Valve/Source-based)

- Running Linux VMs

- Managing H264 Video Surveillance System (16 cameras @ 1080p/24fps)

- Managing & Hosting Video/VoIP Conferences in 1080p (6 people @ 1080p)

- Java Applications (Background Apps; Permanent)

*The computer's secondary tasks would be:*

- Browsing

- Watching 4K Content

Any suggestions or advice before I proceed? I was originally going to wait for Haswell-E to replace the SR2 build, but I'd like an upgrade now.


----------



## Krobar

Do you intend to overclock?
Quote:


> Originally Posted by *Kree*
> 
> I've decided that I'm going to go for the Asus board.
> 
> Here is my current configuration:
> 
> *CPU*: 2x Intel Xeon E5-2697 V2
> *GPU*: 4x AMD Radeon R9 290X's (XFX, Sapphire, MSI & PowerColor)
> *RAM*: 64GB of Hynix DDR3-1600, ECC, CAS = 11-11-11
> *PSU*: Enermax EPM1500EGT (1.5Kw)
> *SSD*: Samsung 840 EVO.
> *MoBo*: Asus Z9PE-D8
> *Cooling*: 2x Zalman Reserator 3's.
> *Case*: Silverstone Raven RV03B.
> 
> *The computer's primary tasks would be*:
> - Running 3x 4K Monitors
> - Encrypting & Hashing (AES256/SHA256/RSA4098/Salsa20)
> - Gaming (Crysis, ArmA 3)
> - Photo Editing
> - Programming
> - Hosting TS3/Mumble Server
> - Hosting Game Servers (Valve/Source-based)
> - Running Linux VMs
> - Managing H264 Video Surveillance System (16 cameras @ 1080p/24fps)
> - Managing & Hosting Video/VoIP Conferences in 1080p (6 people @ 1080p)
> - Java Applications (Background Apps; Permanent)
> 
> *The computer's secondary tasks would be:*
> - Browsing
> - Watching 4K Content
> 
> Any suggestions or advice before I proceed? I was originally going to wait for Haswell-E to replace the SR2 build, but I'd like an upgrade now.


Quote:


> Originally Posted by *lowfat*
> 
> What is point if there are no boards available for it?


The sockets quite similar, theres a


----------



## lowfat

Quote:


> Originally Posted by *Krobar*
> 
> The sockets quite similar, theres a


The socket is exactly the same. Still no boards that support it.


----------



## Krobar

Quote:


> Originally Posted by *lowfat*
> 
> The socket is exactly the same. Still no boards that support it.


I think that was a left over from a quote length limitation in the reply; did not mean to say anything about that.


----------



## TheBlademaster01

Electrically 2011-3 is significantly different from 2011-0 though. It's also keyed differently.


----------



## gurmukh23

when do you think Haswell E will come out? They only recently released the v2s.


----------



## TheBlademaster01

I think Q2/Q3 2014 is what Intel aims for. There are already some ES CPUs around I believe though.


----------



## 364958

I don't intend to over clock. However, I'd love it if it could get to 3GHz.


----------



## psyq3212

Quote:


> Originally Posted by *TheBlademaster01*
> 
> I think Q2/Q3 2014 is what Intel aims for. There are already some ES CPUs around I believe though.


Haswell EP B-samples (ES1) are out for more than 2 months already









However, I don't expect those ES samples to surface on Ebay until the platform is officially launched, as one would need the motherboard and DDR4 RAM - both of which are also only available as development kits right now.


----------



## RoboChimp

Hi guys, long term lurker, new member. I'm a VFX student and as such my specs aren't as good as most of you, but I thought I'd share:
2X Xeon 2650 v1
2X H60 SE
Asus Z9-PE-D8 WS
64GB of DDR3
(32GB 1600 DDR3 Kingston Value RAM + 16GB transcend)
GTX 780
2X Intel 520 SSD in RAID 0 (using Marvell Controller)
2TB WD Black
Cosmos II
Corsair AX760.
Windows 7

I only have 2 issues which you guys may be able to help with;
1. Ever since bios 5206 I get the Error code 66 on the post readout.
2. When I login after standby mode the desktop appear but the system hangs for 10 seconds and recovers.

If any of you know what's going on there, please tell me.


----------



## poulk

Hello guys!
I need your help!
At last i'm assembled PC
asus mb
2x xeon e5 12 core b3
4x 16gb ecc
geforce titan
1kwt PSU
after assembly and connect all the wires, it does not start








power is supplied to the motherboard
POST code on board "00". in manual its nothing say about this post code.
MB dont even "beep" on load
but it start (fan on videocard run)
so that i need to do?
where i need start to find reason of problem?

first that i think its bios (old version). i don't install new. but maybe it's maybe sth else?
BTW how do you think can i use asus bios recovery via usb to install fresh bios? (i scare to kill hardware)


----------



## lowfat

Have you tried pulling the battery and clearing the cmos?


----------



## poulk

Quote:


> Originally Posted by *lowfat*
> 
> Have you tried pulling the battery and clearing the cmos?


no.
but its brand new board (i self open it)


----------



## lowfat

Quote:


> Originally Posted by *poulk*
> 
> no.
> but its brand new board (i self open it)


When I get a 00 I always have to clear the CMOS. And if you are using Ivy Bridge CPUs you will likely need a newer bios than what the board came with. You can buy replacement bios' on eBay.


----------



## poulk

Quote:


> Originally Posted by *lowfat*
> 
> When I get a 00 I always have to clear the CMOS. And if you are using Ivy Bridge CPUs you will likely need a newer bios than what the board came with. You can buy replacement bios' on eBay.


ok i try it. thx
order new flash is to loong for me now








btw that about asus utiltes?
and myh friend have 3930K maybe i try this cpu to load new bios (btw can i run xeon on x79 board? to check cpu do it ok)
UPD. clear cmos don't help








UPD2. live photo (maybe i miss sth then assable)


----------



## ZDngrfld

Quote:


> Originally Posted by *poulk*
> 
> ok i try it. thx
> order new flash is to loong for me now
> 
> 
> 
> 
> 
> 
> 
> 
> btw that about asus utiltes?
> and myh friend have 3930K maybe i try this cpu to load new bios (btw can i run xeon on x79 board? to check cpu do it ok)


You can go both ways. The 3930k should work in the Z9 with non-ecc memory. Depending on the motherboard and it's bios the Xeons should work in X79. I'm betting the bios on your Z9 is too old. You won't be able to flash a new bios without a working processor. Throw that 3930k in there and flash the bios. Follow Asus's directions when you flash... You need to flash a certain version before you can flash the latest version if you're running an old non-CAP based bios.


----------



## poulk

ZDngrfld
thx
i hope "this is it"
tomorrow i try grab the 3930K and make it work.
btw which socket i need to use to run with 1 cpu ?


----------



## anubis1127

The socket furthest from the I/O area.

New CPU coolers came in:





Ahh! So many LEDs, good thing this case has a side panel. The fans that came with the CoolerMaster Seidon 120V are super loud too, those may need to be replaced.


----------



## asusdude24

I use this board and you can't use non xeon processors in it. The only processors that will work in this board are the 2600 series. If you get any processor other than the 2600 xeons, you are wasting your money. You can still try it, but it will not work. The desktop processors will NOT work in a server board, but you can use the xeons in a desktop board using the X79 chipset, but not a desktop processor in a dual processor board.


----------



## asusdude24

You folks here apparently don't read on Asus' website what processors will work with this board. Only the 2600 series xeons will work in this board. I have tried other models in here with regular ddr3 memory just to try it and it will not work. If you spend money on these desktop processors, I guarantee they will not work. Don't bother wasting your time and money on getting them as you will be disappointed when they don't work.


----------



## TheBlademaster01

Did you put it in socket 1? I can't see why it wouldn't work. E5-4600(V2) works as well and isn't listed on Asus' support list either.


----------



## TheBlademaster01

Quote:


> Originally Posted by *anubis1127*
> 
> The socket furthest from the I/O area.
> 
> New CPU coolers came in:
> 
> 
> 
> 
> 
> Ahh! So many LEDs, good thing this case has a side panel. The fans that came with the CoolerMaster Seidon 120V are super loud too, those may need to be replaced.


Looks much better 'nub

I see you've got yourself some new RAM as well that matches your board


----------



## asusdude24

Quote:


> Originally Posted by *ZDngrfld*
> 
> You can go both ways. The 3930k should work in the Z9 with non-ecc memory. Depending on the motherboard and it's bios the Xeons should work in X79. I'm betting the bios on your Z9 is too old. You won't be able to flash a new bios without a working processor. Throw that 3930k in there and flash the bios. Follow Asus's directions when you flash... You need to flash a certain version before you can flash the latest version if you're running an old non-CAP based bios.


The 3930k will not work in this board. Only the 2600 series xeons will. I suggest you go to Asus' website and check the list of the processors that will work. I've tried the desktop versions for the heck of it and they don't work. The reason they don't is because the xeons need the dual QPI links to be able to communicate with each other, features the desktop versions either don't have or are disabled. While these xeons will work in a board using the X79 chipset, the converse is not true.


----------



## lowfat

Quote:


> Originally Posted by *asusdude24*
> 
> You folks here apparently don't read on Asus' website what processors will work with this board. Only the 2600 series xeons will work in this board. I have tried other models in here with regular ddr3 memory just to try it and it will not work. If you spend money on these desktop processors, I guarantee they will not work. Don't bother wasting your time and money on getting them as you will be disappointed when they don't work.


I believe any LGA2011 CPU in CPU1 should work when used by itself.


----------



## anubis1127

Quote:


> Originally Posted by *TheBlademaster01*
> 
> Looks much better 'nub
> 
> I see you've got yourself some new RAM as well that matches your board


Thanks! Yeah, its all in a case, with matching CPU coolers, craziness. Almost like its a high-end workstation.


----------



## asusdude24

Quote:


> Originally Posted by *TheBlademaster01*
> 
> Did you put it in socket 1? I can't see why it wouldn't work. E5-4600(V2) works as well and isn't listed on Asus' support list either.


I would like to see a screenshot of those processors working in this board. If you have them, please provide them.


----------



## asusdude24

Quote:


> Originally Posted by *lowfat*
> 
> I believe any LGA2011 CPU in CPU1 should work when used by itself.


And I disagree. But if you can get it to work (which I doubt) please do provide screenshots if you can. I'd very much like to see this.


----------



## ZDngrfld

http://rog.asus.com/forum/showthread.php?15322-Z9PE-D8-WS-and-I7-3960x
Quote:


> *I just put a Z9PE-D8-WS on my case and an I7 on the CPU1 slot.
> With defaults values on Bios I can boot and installed Windows 7 64*


----------



## asusdude24

Quote:


> Originally Posted by *ZDngrfld*
> 
> http://rog.asus.com/forum/showthread.php?15322-Z9PE-D8-WS-and-I7-3960x


Right. But where is the screenshot of it actually working? People can say what they want in any forum, but until there's a screenshot, I find it hard to believe. Even if it were possible, I'd be curious as to how stable it would be. In theory you should be able to. If you are able, I hope to see screenshots.


----------



## ZDngrfld

Quote:


> Originally Posted by *asusdude24*
> 
> People can say what they want in any forum


This is how I'm seeing you at this moment.

Asus officially supporting it and the board accepting it are two completely different things. How many supported ES processors do you see on Asus's list? The answer is zero, but there's plenty of people using them. In this case @poulk just needs to flash a new bios so he can run the new IVB-E chips. Using a 3930k to facilitate that shouldn't be a problem.


----------



## asusdude24

Quote:


> Originally Posted by *ZDngrfld*
> 
> This is how I'm seeing you at this moment.
> 
> Asus officially supporting it and the board accepting it are two completely different things. How many supported ES processors do you see on Asus's list? The answer is zero, but there's plenty of people using them. In this case @poulk just needs to flash a new bios so he can run the new IVB-E chips. Using a 3930k to facilitate that shouldn't be a problem.


Hope it works for him and that he isn't wasting his time. I've been looking to upgrade to the new v2 series. How are they in terms of overclockability and performance? Haven't see a lot of reviews on this subject.


----------



## ZDngrfld

Quote:


> Originally Posted by *asusdude24*
> 
> Hope it works for him and that he isn't wasting his time. I've been looking to upgrade to the new v2 series. How are they in terms of overclockability and performance? Haven't see a lot of reviews on this subject.


There's a guy who was posting in here not too long ago who was able to get a 114 BCLK on his 2687W V2s. I could get to 104 on my 2650s. I've yet to see what I can get on my V2s. In [email protected] at stock clocks I get a little less PPD than my 4P Opteron setup.


----------



## RoboChimp

Do any of you know what Error 66 means on the post read out?
It seems like I'm getting the cold shoulder here, did I forget to attended an initiation ceremony or something?


----------



## Houseofdreams

Hi guys.

I'm new to this forum but when I found this topic, I've read it entirely for the last couple of days. I ordered this board together with 2 2630 v2 Xeons.

But after reading, I allready know I will be having problems booting because of the v2's. According to the Asus website, I will need at least the 5206 bios version.

My question, and I know there isn't a clearcut answer to this... When you buy this board "today" wich bios version is likely to be installed? I know this depends a lot on how long it has been on the shelve..

My other question is related. I've read a post stating that someone has been able to get the board to boot with a lower bios version, but I can't seem to find that post anymore :-/ This will be then just to be able to update the bios. My only other option will be to replace the bios chip, but removing chips from a brand new board at this price... gives me shivers...

My board will be arriving tommorow so I will know more then.


----------



## Jann

Quote:


> Originally Posted by *RoboChimp*
> 
> Do any of you know what Error 66 means on the post read out?
> It seems like I'm getting the cold shoulder here, did I forget to attended an initiation ceremony or something?


Well judging by the manual it's "CPU DXE initialization is started".
Whatever that means


----------



## RoboChimp

Quote:


> Originally Posted by *Houseofdreams*
> 
> Hi guys.
> 
> I'm new to this forum but when I found this topic, I've read it entirely for the last couple of days. I ordered this board together with 2 2630 v2 Xeons.
> 
> But after reading, I allready know I will be having problems booting because of the v2's. According to the Asus website, I will need at least the 5206 bios version.
> 
> My question, and I know there isn't a clearcut answer to this... When you buy this board "today" wich bios version is likely to be installed? I know this depends a lot on how long it has been on the shelve..
> 
> My other question is related. I've read a post stating that someone has been able to get the board to boot with a lower bios version, but I can't seem to find that post anymore :-/ This will be then just to be able to update the bios. My only other option will be to replace the bios chip, but removing chips from a brand new board at this price... gives me shivers...
> 
> My board will be arriving tommorow so I will know more then.


If it's like most decent boards, it'll read it as an older CPU before the update
Quote:


> Originally Posted by *Houseofdreams*
> 
> Hi guys.
> 
> I'm new to this forum but when I found this topic, I've read it entirely for the last couple of days. I ordered this board together with 2 2630 v2 Xeons.
> 
> But after reading, I allready know I will be having problems booting because of the v2's. According to the Asus website, I will need at least the 5206 bios version.
> 
> My question, and I know there isn't a clearcut answer to this... When you buy this board "today" wich bios version is likely to be installed? I know this depends a lot on how long it has been on the shelve..
> 
> My other question is related. I've read a post stating that someone has been able to get the board to boot with a lower bios version, but I can't seem to find that post anymore :-/ This will be then just to be able to update the bios. My only other option will be to replace the bios chip, but removing chips from a brand new board at this price... gives me shivers...
> 
> My board will be arriving tommorow so I will know more then.


Must be a glitch in the bios update then. I've had it since the lastest update.
Quote:


> Originally Posted by *poulk*
> 
> ok i try it. thx
> order new flash is to loong for me now
> 
> 
> 
> 
> 
> 
> 
> 
> btw that about asus utiltes?
> and myh friend have 3930K maybe i try this cpu to load new bios (btw can i run xeon on x79 board? to check cpu do it ok)
> UPD. clear cmos don't help
> 
> 
> 
> 
> 
> 
> 
> 
> UPD2. live photo (maybe i miss sth then assable)


Looks a bit like my rig:







Quote:


> Originally Posted by *anubis1127*
> 
> The socket furthest from the I/O area.
> 
> New CPU coolers came in:
> 
> 
> 
> 
> 
> Ahh! So many LEDs, good thing this case has a side panel. The fans that came with the CoolerMaster Seidon 120V are super loud too, those may need to be replaced.


I really like what you did with that







, I'm going to revamp my rig and do a similar blue theme.


----------



## RoboChimp

-snip- Accidental double post.


----------



## poulk

guys maybe some one can share link on ebay, where i can find flash bios with latest bios install ?


----------



## gurmukh23

Really sad that in this day and age we only have one decent dual socket mobo! Can EVGA, Gigabyte and Asus wake up!


----------



## anubis1127

Evga tried to make one. It was just so terrible they gave up.

Gigabyte has a couple dual LGA 2011 boards but they are geared toward server use.


----------



## arvidab

ASRock also have some dual 2011 boards, they are also geared towards servers instead of workstation/desktop.


----------



## RickSFO

My system locks up at random, and it's making me crazy... Also making it hard to do long optimization runs which I need for my work.









It happens once a day or so. Usually the screen just freezes, and it locks up hard. I have to power cycle the system.

I have:

Dual 2687W. They're engineering samples. B0, I think.
Bios 0703
GSkill RipjawsZ F3-12800CL9Q-16GBZL
AMD Firepro card

Questions:

1) That memory is non-ECC. Should I get ECC memory?

2) Will newer BIOS have any issues with my ES chips?

3) This interesting comment is on the Asus website:

BIOS 5304 for Z9PE-D8 WS released to public
1. Support Windows 2012 R2.
2. Improve compatibility for add-on VGAs while 2 CPUs installed.
3. Improve compatibility for add-on VGAs with Ivy-Bridge CPUs.

So, by improved compatibility do they mean stability???

Any help here is much appreciated. This is making me crazy.

Thanks,

Rick


----------



## poulk

i think im find that i need (http://www.ebay.com/itm/The-Latest-Version-BIOS-CHIP-for-ASUS-Z9PE-D8-WS-/131040233035?pt=US_Motherboard_Components&hash=item1e829b464b#viTabs_0)
but which bios i need to flash? latest avalible on asus suport download?
(remind you
its for 2x e5 12c b3 cpus)


----------



## Houseofdreams

Quote:


> Originally Posted by *poulk*
> 
> i think im find that i need (http://www.ebay.com/itm/The-Latest-Version-BIOS-CHIP-for-ASUS-Z9PE-D8-WS-/131040233035?pt=US_Motherboard_Components&hash=item1e829b464b#viTabs_0)
> but which bios i need to flash? latest avalible on asus suport download?
> (remind you
> its for 2x e5 12c b3 cpus)


Just have a look on the CPU support page on the Asus website. Find your specific CPU type and behind it you will see the minimum bios version wich will support your cpu's.


----------



## poulk

Quote:


> Originally Posted by *Houseofdreams*
> 
> Just have a look on the CPU support page on the Asus website. Find your specific CPU type and behind it you will see the minimum bios version wich will support your cpu's.


so last one be best choice?


----------



## Houseofdreams

Quote:


> Originally Posted by *poulk*
> 
> so last one be best choice?


Maybe a previous bios version could also work, but my bet will be to try the latest one.. Watch out for the warnings on the bios updates page:

*[Warning]
*If your BIOS version is 0703 or older: Go to BIOS-Utilities to download and flash the BIOS to version 3109 firstly before updating the BIOS.*

Otherwise you could be stuck with a brick and your only solution than is to buy a preflashed bios from Ebay (som somewhere else)


----------



## poulk

Quote:


> Originally Posted by *Houseofdreams*
> 
> Maybe a previous bios version could also work, but my bet will be to try the latest one.. Watch out for the warnings on the bios updates page:
> 
> *[Warning]
> *If your BIOS version is 0703 or older: Go to BIOS-Utilities to download and flash the BIOS to version 3109 firstly before updating the BIOS.*
> 
> Otherwise you could be stuck with a brick and your only solution than is to buy a preflashed bios from Ebay (som somewhere else)


hehe i write it before (when ask) its for new preflash bios


----------



## Houseofdreams

Quote:


> Originally Posted by *poulk*
> 
> hehe i write it before (when ask) its for new preflash bios


Sorry







I'm currently reading multiple forum posts at once. Hrd to keep track


----------



## NapalmV5

Quote:


> Originally Posted by *LeMakisar*
> 
> Exact same problem with Portal 2, I've spent a lot of time with Valve support but it was useless ... luckily I still have another computer so I could finish this wonderful game
> 
> 
> 
> 
> 
> 
> 
> 
> 
> Deus Ex : HR also has issues but you just have to restrict the steam.exe to a few cores and it just works.
> 
> GRID doesn't work either (a shame :/)
> 
> If someone has information to make these work that would be nice indeed.


damn.. all the other valve games ive tried so far work/load up fine.. i think ms visual c++ has something to do with it but why portal 2 refuses to load - i dont know what to make of it.. ill try their support

thanks man i appreciate it!


----------



## anubis1127

I just updated my Z9PE-D8 WS board to 3109, I couldn't boot with the HD 7950 I just got on the old bios I had on it.


----------



## TheBlademaster01

And, and? Did you brick anything yet







?


----------



## anubis1127

Quote:


> Originally Posted by *TheBlademaster01*
> 
> And, and? Did you brick anything yet
> 
> 
> 
> 
> 
> 
> 
> 
> ?


Haha, nope. I posted that from within Windows 7 on the Z9PE-D8 board. Here is a CPU-Z:



One thing that is alarming is the Core Voltage CPU-Z reads for the CPUs, that doesn't look correct, at least I hope the board isn't doing that to them.


----------



## TheBlademaster01

You should notice that on power consumption and temperatures under load if that's true.

I use HWInfo64 btw.


----------



## anubis1127

Quote:


> Originally Posted by *TheBlademaster01*
> 
> You should notice that on power consumption and temperatures under load if that's true.
> 
> I use HWInfo64 btw.


Yeah, max temps even when I was folding on the CPUs is under 50C on every core, under 45C on most, so I can't think it is correct.

I just downloaded it, thanks for the tip, so far the max core voltage it has monitored is 1.175V, which sounds more reasonable.


----------



## TheBlademaster01

Yeah, CPU-Z doesn't even see your 2nd CPU IIRC lol. I stopped using it, HWInfo is much more detailed (even making GPU-Z kind of irrelevant IMO. But I might not be the best person to judge this since I have a boring GPU


----------



## anubis1127

Quote:


> Originally Posted by *TheBlademaster01*
> 
> Yeah, CPU-Z doesn't even see your 2nd CPU IIRC lol. I stopped using it, HWInfo is much more detailed (even making GPU-Z kind of irrelevant IMO. But I might not be the best person to judge this since I have a boring GPU


Yeah, HWInfo has a ton of info in it, I've installed it in the past, I don't know why I always gravitate toward CPU-Z. For this PC I'm definitely going to continue using HWInfo.


----------



## arvidab

Yea, mine show some crazy voltage in CPU-Z too, HWinfo showed a reasonable value and I chose to trust that one instead.


----------



## vtwinracersp2

case fan PWM not working

updated bios from 3109 to 5304.

now the case fan pwm just goes to max rpm.

has this happened to anyone else? any ideas or solutions?


----------



## vtwinracersp2

what is the bios chip model if i want to replace the one i have?

i see that asus has some bios chips in their store, but i'm not sure i'm finding something for the Z9PE-D8 WS.


----------



## anubis1127

Quote:


> Originally Posted by *vtwinracersp2*
> 
> what is the bios chip model if i want to replace the one i have?
> 
> i see that asus has some bios chips in their store, but i'm not sure i'm finding something for the Z9PE-D8 WS.


http://us.estore.asus.com/index.php?l=product_detail&p=5603

Then select Z9PE-D8 WS from the Model drop down.


----------



## 364958

I have a question. Since the new Mac Pro uses a 12 core Xeon, and the Z9PE-D8 has lots of faults/problems and horrible tech support from Asus, would any of you just recommend me going for the new Mac Pro?


----------



## ZDngrfld

Quote:


> Originally Posted by *Kree*
> 
> I have a question. Since the new Mac Pro uses a 12 core Xeon, and the Z9PE-D8 has lots of faults/problems and horrible tech support from Asus, would any of you just recommend me going for the new Mac Pro?


Do you want to run OSX? If so, yes... Every hackintosh I've used just isn't the same... There's always something that isn't quite right. My brother fights with his every. single. day.


----------



## 364958

Quote:


> Originally Posted by *ZDngrfld*
> 
> Do you want to run OSX? If so, yes... Every hackintosh I've used just isn't the same... There's always something that isn't quite right. My brother fights with his every. single. day.


Yes. My primary operating system will be OS X.


----------



## poulk

hello guys
i need serious help
today i flash new bios(latest one) with 3930k
after this system start and begin load bios. but it's stops on post code error "5A" (internal cpu eror)
i remind i have 2x 12core e5 xeon b3 stepping ES (at least this seller say)
i use 1x 16gb ddr3 ecc ram module for each cpu.
that i do wrong? and that i need to do?
maybe i need do some bios changes before install xeon?
or need try sth else? may it be memory problem?
btw seller have notice this
Quote:


> BIOS setting, you may need to disable C6 C7, a lot of Power supply especially the older ones , do not supply the new intel 0A PSU load. It
> 
> will freeze the computer. in the BIOS it is called 'Lowest CPU Idle Power Setting' and then on the PSU support list they are called "12V2
> 
> Min-load 0A"


can this have some importance in this situation?


----------



## poulk

ok.
i install 2 more ram modules (now its 4)
after this system started up and I was able to go into the BIOS setup.
but after I went into the BIOS after a few seconds the screen went blank. after that the system no longer run. on the motherboard does not work even post code screen! but the board is turned on and supplies power to the video card!
motherboard died? died BIOS? what to do and what to try?


----------



## thespecialist

If need u need two Cpu at once, you're either running a at home server or u want to hack through the NSA, LOL


----------



## brucethemoose

Quote:


> Originally Posted by *thespecialist*
> 
> Im not familar with this board but if u can run 2 cpus at once HOLY CRAP. This is come hacker/server lvl equipment son


There are actually some 8 CPU Xeon motherboards coming out, so you can have 120 physical ivy bridge cores in a single machine... That's like 20 i7s with coherent memory


----------



## asusdude24

Quote:


> Originally Posted by *Kree*
> 
> Yes. My primary operating system will be OS X.


You're going to use a girlie man operating system?


----------



## asusdude24

Quote:


> Originally Posted by *thespecialist*
> 
> If need u need two Cpu at once, you're either running a at home server or u want to hack through the NSA, LOL


Aww darn you just exposed what I use my computer for.







The NSA won't be pleased







(kidding)


----------



## thespecialist

Quote:


> Originally Posted by *brucethemoose*
> 
> There are actually some 8 CPU Xeon motherboards coming out, so you can have 120 physical ivy bridge cores in a single machine... That's like 20 i7s with coherent memory


let me just spend 5k on a new build and see what every intelligence agency is doing, daily








*joke


----------



## poulk

Quote:


> Originally Posted by *poulk*
> 
> ok.
> i install 2 more ram modules (now its 4)
> after this system started up and I was able to go into the BIOS setup.
> but after I went into the BIOS after a few seconds the screen went blank. after that the system no longer run. on the motherboard does not work even post code screen! but the board is turned on and supplies power to the video card!
> motherboard died? died BIOS? what to do and what to try?


no one can't say nothing helpful?


----------



## jiahuolila

I was also a little concerned with how stiff those Corsair hoses are! It made me a touch apprehensive that the gymnastics required to mount them might possibly weaken them for later cracking. I'm sure they're pretty reliable tho. Cool running too. I settled for air-cooling with the Dynatron R17s. They fit nicely with hi-rise RAM and work just fine. CPU 2 is the hottest due to airflow config and it doesn't get over 50C with heavy rendering or 3DMark11 tests


----------



## poulk

heh i find how fix problem, my self.
but now, some time when start system, i get same post code "5A" internal cpu problem.
how do you people.
where i need to dig? it's cpu problem or motherboard?


Spoiler: Warning: Spoiler!



i don't hope to get the answer, but i try


----------



## TubbeAh

Quote:


> Originally Posted by *vtwinracersp2*
> 
> case fan PWM not working
> 
> updated bios from 3109 to 5304.
> 
> now the case fan pwm just goes to max rpm.
> 
> has this happened to anyone else? any ideas or solutions?


Updated from 5004 to 5304 and I have the same problem... fans roaring om max rpm. ***? Even tried manually setting the speed, but no difference.


----------



## ocvn

Quote:


> Originally Posted by *poulk*
> 
> heh i find how fix problem, my self.
> but now, some time when start system, i get same post code "5A" internal cpu problem.
> how do you people.
> where i need to dig? it's cpu problem or motherboard?
> 
> 
> Spoiler: Warning: Spoiler!
> 
> 
> 
> i don't hope to get the answer, but i try


5A might be your hdd/sdd problem


----------



## jfrey

Hello everyone,

Joining this forum, I hope one of you experts will be able guide me through my *annoying* problem.

Short version: no matter what I plug or not, the onboard power led/button of my Z9PE-D8 blinks and... that's it. Nothing moves or lights up (besides that hypnotic flashing light).

I have a corsair ax1200i PSU and two xeon E5-2687W v2. All the material is brand new, I was planning to build one hell of a computer for new year's eve -- silly me.

I tested the PSU on another (and way less powerfull) computer, it works. I also tried the PSU of that computer (another corsair but only 400W) with the motherboard: same thing.

Long version: yesterday I happily assembled everything, mounting both CPU, my noctua fans, RAM, graphic card. On a table, not in my case. I switched-on the PSU and... the system started to boot right away. Nothing on screen though, the diagnostic leds stopped with a "6C" code. Because yesterday I did not receive my EEC RAM yet, I had borrowed two sticks from my working station, putting 2GB modules in A1 and E1 as advised by the manual. I checked on Internet and understood that the "6C" code could be related to an incompatible RAM modules.

I did not want to give up yet, I started to remove one module, disconnect the power of the second CPU, put back one module, remove the graphic card, move the graphic card, etc. I had not many different things to do and yet something strange began to occur. First, when I switched-on the PSU, sometimes, instead of booting and freeze on "6C", the motherboard hanged with no reason after one or two seconds. Switch on/off a couple times and it booted again, straight to "6C". I made some more attempts (I do not remember exactly what I did with the RAM and GPU, but nothing silly). But then, even more oddly, the motherboard on some occasions seemed to boot, but with nothing appearing on qleds. Just the fan spinning (the only upside: 4 noctua fans + 1 MSI GTX650ti boost twin frozr = very quiet).

So within few minutes I came from "6C error code" to "no error code". Some more restarts/false starts and it happened: the blinking power led. I tried to put things back exactly as it was the first time, when I still got this (beautiful) 6C error code, but no matter what, I got nothing more than blinks, blinks and blinks. *Then* I started to panic. I remove the fans, remove one CPU, two CPU (!), plugged nothing but the 24 pins power cable: the same. I also tried to clear the RTC (with or without the battery, just got a still red power the time I forgot to put the jumper back).

Today I received my ECC registered and (supposedly) compatible crucial RAM modules: of course it did nothing to help. Now you know the whole story. There's only one thing I could add: the corsair PSU has some kind of "auto diagnose" button. When there is no cable attached to it, it starts the fan and a green light turns on. If I press this button while the PSU is connected to the motherboard, I hear a "click" each time the power led blink. As if for a fraction second it was gonna boot. And from time to times there's even the fans which start to move, slightly, if I connected them.

Well, if you have any idea on what's going on, I'm listening. I already thank you if you read my whole post







Honestly, this thing drives me crazy. And not only because I have around 6k dollars of useless hardware laying on my table. I just do not understand, and *this* is the worst.

Nothing seems bent or damage -- I did so little besides plunging/unpluging cables between the PSU and the motherboard than I don't see how I could have broken anything.

Short question for the people who have no clue but own a Z9PE-D8: what happens if you plug nothing but the 24 pin cable? Can you boot "a little" ? Because if your power led *does not* blink, then the culprit is either the MB or the PSU; I don't have to bother anymore with CPU/RAM/GPU etc.

Ahem, for a first post I guess I wrote enough







Happy new year for the ones who have better things to do this time of the year than hanging on this topic


----------



## ZDngrfld

@jfrey
First off, I'm assuming you're doing all of this with both 8-pin EPS plugged in? This board won't boot (regardless of how many processors you have installed) without both 8-pin plugged in. You don't need the EZ-PLUG 4-pin molex. So this answers your question about the 24-pin power cable. Since you're using the Ivy Bridge V2s you're going to need to be sure you're on bios version 5103 at the minimum. I'm wondering if this is the problem... You might want to install a Sandy Bridge chip if you have one laying around somewhere and see if it lets you boot. If it does, check your bios version. If you're before 0703, you'll need to update to 3109 and then you can update to the latest version, which is 5304.


----------



## jfrey

Thanks for the quick reply.

Yes, I tried with both 8-pin plugged in. I do not have the opportunity to try other CPUs at the moment. I don't know which BIOS version I have since I never made it to POST. But what amazes me is that, regardless of the BIOS version or the CPU type, it worked the first time! Well, kind of, I had an error code, but at least I did not have this damn blinking! When you say the system wouldn't boot without the 8-pin cables, do you mean *nothing* would happen or could it the cause of the "power blinking" symptom?


----------



## ZDngrfld

If it does anything, it will power cycle on and off. You might see the fans spin up momentarily and then power back off. It could be your bios version or it could be the board needs RMAed. I had to get an RMA before I could do anything. First one was DOA


----------



## ikem

finishing up my z9 build.


----------



## anubis1127

@ikem Looks great.

@NapalmV5 I finally got around to do some gaming on my Z9PE-D8 WS board today, using Win7, and a HD 7950. Tomb Raider would not start for me, set affinity to use only 1 CPU, then it worked. Just Cause 2 worked out of the box. I'll try Witcher 2, and Portal 2 next, and report back.


----------



## Houseofdreams

Hi....

I Finally got my parts, 1 cpu instead of 2, but hey, I can start building...

Bios version installed from factory was 5103, but it did recognize my CPU, when the support list stated it should be 5206, so for others that want to know this, even a lower bios version will boot, so you can update it..

My problem:

I have for testing purposes installed a single 1TB hard drive on the blue sata connectors, but I cannot see it when installing windows 7. I have been looking in the bios but I can only find some cryptic "IBA GE Slot 0600 v1535" and a "IBA GE Slot 0700 v1535" as the boot devices..

Nowhere is a "link" found to mu actual hard drive.

Anyone know what I'm doing wrong (or forgetting to do) ?

Thanx!

EDIT: I have a brand new power supply (AX760) but it looks like the drives aren't getting power? I know power is going trough because of the watercooling unit that's getting its power from the same lead..

Very strange...


----------



## Krobar

Quote:


> Originally Posted by *asusdude24*
> 
> I use this board and you can't use non xeon processors in it. The only processors that will work in this board are the 2600 series. If you get any processor other than the 2600 xeons, you are wasting your money. You can still try it, but it will not work. The desktop processors will NOT work in a server board, but you can use the xeons in a desktop board using the X79 chipset, but not a desktop processor in a dual processor board.


Single CPU I7 LGA2011 does work in a lot of Dual LGA2011 in single cpu configuration, it depends on the bios.

Supported and works are very different things, to flash the bios you just need "works".


----------



## singh_shady

I'd like to join the elite club of dual socket boards please.









Got my Z9PE-D8 the other day and recieved my CPU's today, not without problems though.

I have these 2 bad boys, which after a quick google search of the QDU4 code comes up as the e5-2640 v2. But the markings are different on each? one says ES and the other says ES2? whats up with that?
Can they work together, because im not getting it to post.
Straight away with 00 error on the LED readout?? any ideas? tried swapping cpus, single cpu only in the right socket and removing and replacing ram etc.


----------



## TheBlademaster01

You need BIOS version 5206 or later for it to post with non ECC RAM. Make sure you have that.


----------



## singh_shady

i have bought Hynix EEC 4gb sticks, x6 in total, with 2 more to come for 32gb.
Ive placed the memory in every configuration and tried single sticks too, but to no avail.

Ive ordered a bios chip with the 5306 bios on it, hopefully that lets it boot.


----------



## singh_shady

Ok, got the bios chip put in with the latest version and still no post, stops at error code 'AC'

Any ideas?


----------



## TheBlademaster01

Not sure, I encountered issues with my board yesterday as well...

Stops at AE

I can't even find an appropriate Q code for it.


----------



## singh_shady

Well, i swapped the CPU's in the sockets and that fixed it. Booted right up







. Going to get my ES swapped for a ES2 so they match and hopefully fixes that issue.
BUT I'm getting crashes in windows every 5mins or so in my existing windows8.1 raid0 array from my previous rig. Would have been nice to just plug and play but i guess it wont play nice without having a fresh install, thats what im hoping the problem is anyway.
the first time it crashed, windows gave me the BSOD so had a code to troubleshoot, but doesn't display anything like that since then, straight freezes up or reboots. First BSOD gave dpc_watchdog_violation. Something to do with the SATA im guessing.

Time for a evening of reinstalling windows, drivers and all software, should take the rest of the day up, better put the kettle on.


----------



## TheBlademaster01

Yeah, I think I might swap the chips as well









Getting pretty tired of this atm though


----------



## singh_shady

Yh, im considering just going back to a single socket myself and i've only had it working for a day lol
Im getting random lockups with that DPC_watchdog_violation BSOD

Could be to do with my ssd raid0 but its worked fine for me previously, fiddling with bios right now and hoping i can get it to stop crashing on me. Maybe to do with the ES2 and ES cpu not liking each other.

Blademaster, You seem to have had this board for a long time, looking back on this thread? how has the board been for you in general?


----------



## TheBlademaster01

Well after having a conversation with someone who tried various boards and chips I think it's mostly bad quality check by Asus. Also slow in addressing certain things while breaking other features.

Then there are just flaws in certain ES chips. My B0 chips for example have issues with PCI-E 3.0 GPUs and namely the .cap BIOSes (3xxx and later).

My system however was perfectly fine on the non .cap BIOSes and PCI-E 2.0 and older GPUs. For some reason the board tried to force 3.0 when detected on the older BIOSes which made it not recognize cards and/or displays since the board didn't support it at the time. Something like that, or it was clocking the PCI-E bus that resulted in failure (downclocking 3.0 x16 to x1 when idle for example). It took them about 4 months in the time to address it and the releases had minor bugs in them as well in between like bricking the manual bClk setting or unstable memory etc.

I think that if you have proper chips, a working board (no DOA) and a BIOS version that works for you it is a stellar board. If you have exotic hardware that isn't supported (NICs, RAID controllers etc.) Asus support is pretty worthless and chances are it will never work (or end up being broken for a long time).


----------



## singh_shady

To be honest, I'm more of an onboard features kind of guy lol Normally end up just having a single graphics card and nothing else occupied in the slots.
It's primarily going to be for Maya and After Effects work, so hoping it wont let me down with that.

Seems to be ok for now though... I've taken out the ES2 CPU from the second socket, and running just the single configuration and its been stable for an hour now, seems to have fixed it.
Maybe its the CPU then, ill get it swapped to match, so either x2 ES2 cpus or x2 ES's.


----------



## TheBlademaster01

Yeah, I used it for Cinema4D, bit of mining and [email protected] Excellent performer in all of the applications and efficient. And like I said, it worked excellently with my 470 just my 780Ti


----------



## singh_shady

what kind of problems with the 780ti? No post at all?
running a r9 290 on the board right now, haven't got around to stressing it yet.


----------



## TheBlademaster01

Intermittent posting and only having display via HDMI on my TV. You should have no problems on Ivy and that BIOS revision. When it posts it works stellar as well though in PCI-E 2.0 mode.


----------



## anubis1127

Quote:


> Originally Posted by *TheBlademaster01*
> 
> Not sure, I encountered issues with my board yesterday as well...
> 
> Stops at AE
> 
> I can't even find an appropriate Q code for it.


I have gotten AE in the past, usually when Win7 hasn't started successfully.


----------



## DualFiz

Quote:


> Originally Posted by *singh_shady*
> 
> To be honest, I'm more of an onboard features kind of guy lol Normally end up just having a single graphics card and nothing else occupied in the slots.
> It's primarily going to be for Maya and After Effects work, so hoping it wont let me down with that.
> 
> Seems to be ok for now though... I've taken out the ES2 CPU from the second socket, and running just the single configuration and its been stable for an hour now, seems to have fixed it.
> Maybe its the CPU then, ill get it swapped to match, so either x2 ES2 cpus or x2 ES's.


Hmm... what sort of RAM are you using in your build? ie, brand? ECC? Registered? Buffered? ...and how many slots are you occupying?

I only ask as stability with two CPU's might be reflective of RAM compatibility issues and not ES compatibility problems.


----------



## singh_shady

Hynix 4GB 2RX4 PC3 10600R RAM P/N HMT151R7BFR4C-H9 (32gb total)

Went into power management features for the cpu in the bios and disabled speedstep among other power saving features and it fixed the BSOD and crashing issues







Going to cycle back through some of the options later on to narrow down which specific option is the culprit.
These are a strange breed of CPU lol

Well now it runs stable all the time. Except when the computer is left idle and goes to sleep, then it wont wake up again. Also it wont turbo up to x23 multiplier for 2.3ghz, runs around 2ghz and occasionaly it bumps up to 2.2ghz for a second and back down while IDLING







according to CPU-Z.
Under load it stays rock steady at 2ghz only?


----------



## DualFiz

Quote:


> Originally Posted by *singh_shady*
> 
> Hynix 4GB 2RX4 PC3 10600R RAM P/N HMT151R7BFR4C-H9 (32gb total)
> 
> Went into power management features for the cpu in the bios and disabled speedstep among other power saving features and it fixed the BSOD and crashing issues
> 
> 
> 
> 
> 
> 
> 
> Going to cycle back through some of the options later on to narrow down which specific option is the culprit.
> These are a strange breed of CPU lol
> 
> Well now it runs stable all the time. Except when the computer is left idle and goes to sleep, then it wont wake up again. Also it wont turbo up to x23 multiplier for 2.3ghz, runs around 2ghz and occasionaly it bumps up to 2.2ghz for a second and back down while IDLING
> 
> 
> 
> 
> 
> 
> 
> according to CPU-Z.
> Under load it stays rock steady at 2ghz only?


Cool cool... yeah, I know with a LOT of the earlier ES CPU's, sleep states are a huge problem, so much so the only real solution is to disable them entirely. At least, that's what I've been told.

The turbo feature is probably working correctly... well, maybe. When you are testing this function are you doing so with a multithreaded application? Or something that might use a single core?

The max turbo speed is really designed for single core operation... when multiple cores are used, the turbo limit drops.

BUT, you probably already know that, so I'm not helping at all.


----------



## singh_shady

didn't know that specifically, assumed all cores could turbo. But i did try both incase of that anyway, used prime95 with all threads active and superpi.

Made some head way in finding out the exact issue in the bios, it seems under SpeedStep 'enabled', there is then a 'P-State' option that appears, that has to be set to 'SW_any'.
Then C-State limit has to be set to 'C7' since the default 'no limit' selection causes the dreaded crashing and reboots, aswell as the P-State being on 'HW_all'.

well that was a productive and fulfilling troubleshooting session, hopefully helps someone else out.


----------



## ZDngrfld

You can have all the cores hit the turbo. They just have a lower limit than a single core. It's usually a couple hundred MHz. It varies per model of processor.


----------



## Shane Daly

Hi All

I'm finishing a build with this board and 2x12core Xeons, happy to share my experiences here.

My specs are;
Caselabs TX-10
1 x Z9
2 x v2 2.7 Xeon
8 x 16GB (128GB) 1600Mhz RAM Kingston (per Asus spec)
1 x GTX Titan
1 x Decklink 4K
1 x Areca 1882ix-24 4G RAM
24 x 4TB 7200rpm HDD in RAID6 = 88TB avail, dual redundancy
2 x 512GB Samsung 840 in RAID0 = system drive

I cooled with 2xCorsair H110 which were tricky to install in the case but keep the Xeons at around 32C which is great.

My problem now is how to create the SSD RAID0 on the Intel RAID and have Windows see the disk - I can only run the LSI RAID even though I have moved the jumper over. In LSI I managed to stripe the
drives into 0 but Windows cannot see it to write the OS to?

Also all the SATA2 connectors along the bottom edge don't seem to work, I have my DVD drive currently in one of the Marvell ports.

Ideally I want the 2xSSD in the fastest RAID0 possible out of Intel, LSI and Marvell.

Any ideas? Cheers

Shane


----------



## Shane Daly

Quote:


> Originally Posted by *anubis1127*
> 
> I tried loosening the CPU cooler to the point where its barely tightened, but no change. I'm down to testing with just one CPU, and one dimm slot filled, and still no luck. I've tried unregistered dimms that are on the QVL, and two sticks of Samsung that isn't on the list, and neither made a difference.
> 
> I'm starting to wonder if my ES chips aren't compatible with the board, or if there is just an issue with the board. Unfortunately I don't have any other 2011 Xeons to test with the board. The CPUs work fine in my x79 desktop motherboard.


I received my 3rd board today - the new original was DOA and the refurb replacement had issues so now this one. 00 code on turn on, Asus tech support talked me through switching the actual BIOS chip with my 2nd (as yet unreturned) board because the BIOS liked my Xeon 12 cores.

Take away = it might be a bios compatibility issue with your Xeons?


----------



## Shane Daly

Quote:


> Originally Posted by *Donkey1514*
> 
> Anyone gotten a Q code of "00"? I bought some 2680 ES chips and installed them and when I try to boot all I get is that code..... I double checked all connections and cleared the CMOS several hundred tmes and nothing works. Any help is appreciated.


I had this today on my 3rd RMA board - it's a BIOS conflict. ASUS tech talked me through switching the old dead board BIOS (compatible with the Xeons) with the incompatible BIOS from my new board.

Worked a treat









Investigate BIOS conflict.. Good luck


----------



## Shane Daly

Quote:


> Originally Posted by *lloyd mcclendon*
> 
> ^ agreed
> 
> I would say to avoid use of the Marvell controller. As usual and just as I expected, when that is enabled it breaks the VT-D / IOMMU support since their firmware is *ALWAYS OUT OF SPEC* just as it was on my last ASUS board. So if you need to use VT-D, you have to disable the Marvell controller in the bios. I believe the SSD caching runs through there. As I already eluded to, marvell sucks. period.
> 
> SSDs are getting cheap enough and if you can afford this board with 2 xeons, you probably can eventually afford 4 or 8 decent sized SSDs for a RAID0 or RAID10 setup. Forget spinning drives unless you need 4 of the 2 or 3 TB ones right. In any case you should be able to keep your core files on the SSD and your 'data' on the spinning drives, and most likely the caching wouldn't do a whole heck of a lot anyway given the IO usage patterns.
> 
> You could also set the same thing up using software raid with --write-behind and --write-mostly and we've already discussed why software raid is *almost always better than hardware raid. flame on
> 
> apologies in advance for my tone, I'm about to throw this whole machine out the window. It's as if Murphy himself has moved into the case. (none of these problems are related to the board itself thankfully, it's really nvidia...)


I'm at this point now - 2 x SSD which I want to place in RAID0 software. Would that be the Intel? If so how to get there - I moved the switcher but it still only sees the LSI which I don't want to use..

Any advice will be grateful received, thanks

Shane


----------



## x86box

This board runs osx great. I have x4 of these boards with x2 complete builds. 10.9.1 is easier than ever to install, and I've been running osx on a PC since before Apple sold the first intel laptop. It's a cakewalk now. Head over to insanelymac to see the latest improvements with Chameleon and even more difficult Bios mods with UEFI boot loader and fakesmc injected.

I'm using these boards for high end Post Production. They are rock solid. But I'm still on 3109 with x2 2680 v1.


----------



## Shane Daly

Hey x86box

I'm also building my machine for high end video work - you sound more experienced than me so I'd appreciate any advice.

I want a super stable system, obv, to run DaVinci Resolve. So far so good, my next decision is how to configure the 2 x 512GB 840 SSDs?

Possibilities are;
1 - Intel RAID0 1TB SSD system drive w Windows7
2 - as above but partitioned for Windows7 / OSX Mavericks
3 - buy another 2 SSD and configure as per 2 but in RAID0+1
4 - Have separate drives on SATA3 with Windows7 on one and OSX on the other

From this thread I have learned to avoid the Marvell and any hardware RAIDs, hence me looking into the Intel solution.

I have moved the jumper over but still only get to access LSI - ctrl + I fails, only ctrl + M works to access LSI.

Help!









Thanks, Shane


----------



## x86box

Chameleon is easier to Configure if each OS is on a separate drive. Just plug both into the two blue 6G ports(the ones all the way to the right if you are looking to the rear I/O ports) use the AHCI driver for windows 7 so you can use chameleons boot menu to load the Os of your choice. Although win7 may install it by default. I just use different ssd drives for each OS, doesn't harm my performance in any way. I use my systems for Resolve, AE, Scratch, and Colorfront.

Also, under power management I use Performance settings, and keep PCI clock at 32 to get the best render speeds on my Titan.

Now that Mavericks supports Power Management for 2011 socket it's very easy to run without NullCPUPM. Since I'm running ColorFront, I use Full Performance settings in Bios and force multiplier to x27 on 2680.


----------



## singh_shady

Quote:


> Originally Posted by *Shane Daly*
> 
> Hi All
> 
> I'm finishing a build with this board and 2x12core Xeons, happy to share my experiences here.
> 
> My specs are;
> Caselabs TX-10
> 1 x Z9
> 2 x v2 2.7 Xeon
> 8 x 16GB (128GB) 1600Mhz RAM Kingston (per Asus spec)
> 1 x GTX Titan
> 1 x Decklink 4K
> 1 x Areca 1882ix-24 4G RAM
> 24 x 4TB 7200rpm HDD in RAID6 = 88TB avail, dual redundancy
> 2 x 512GB Samsung 840 in RAID0 = system drive


Holy Guacamole


----------



## NapalmV5

Quote:


> Originally Posted by *anubis1127*
> 
> @ikem
> Looks great.
> 
> @NapalmV5
> I finally got around to do some gaming on my Z9PE-D8 WS board today, using Win7, and a HD 7950. Tomb Raider would not start for me, *set affinity to use only 1 CPU, then it worked*. Just Cause 2 worked out of the box. I'll try Witcher 2, and Portal 2 next, and report back.


thanks.. forget that - everything should work regardless of affinity

i disabled hyper-threading and now portal 2/witcher 2/tomb raider works on both cpus

just cause 2 works on gtx 580 + ht enabled but not on r9 290.. whats funny is that i was able to run it once on r9 290 when messing around with ms visual c++ installing old ones one by one


----------



## Shane Daly

Currently stuck trying to install Win7 on a Samsung 840 Pro SSD, Win setup won't recognize the drive or partition - any ideas?! Google solutions all not working. Stuck!


----------



## NapalmV5

bad win7 ?

6x samsung 840 pro @ pch on win7 here - no issues and never had any regardless of os


----------



## Shane Daly

Brand new install disk, works fine. Problem seem to be some kind of SATA driver MoBo issue? Tried loading drivers from Asus site during install, no joy.

Tried same drive on Marvell controller, same issues.

BIOS sees the SSD correctly, everything seems 100% until I click to install W7 onto the SSD at the last stage, next thing that should happen is the install begins.

Totally stumped!


----------



## singh_shady

are you running raid on the intel sata3 (6gb/s) ports which are blue?


----------



## ignoramus

i know i had to install windows unraided on a single HDD completely and do some updates before raid came up as an option. i then had to reinstall windows in raid. however i have no idea which raid i use, which sata ports my HDDs are plugged into or the capital of france. hopefully thats helpful....


----------



## singh_shady

Quote:


> Originally Posted by *ignoramus*
> 
> i know i had to install windows unraided on a single HDD completely and do some updates before raid came up as an option. i then had to reinstall windows in raid. however i have no idea which raid i use, which sata ports my HDDs are plugged into or the capital of france. hopefully thats helpful....


im bad with sarcasm so ill assume your serious...in which case google is just a stones throw away my friend.


----------



## Shane Daly

They were on the Marvell ports because they didn't show on the blue.


----------



## Shane Daly

Thanks for the help everyone, turns out I'm a moron and leaving my RAID card in caused the problem. As soon as it was out Windows installed. She's stable now on Windows8, nice low temps all round.

Now if I could just get the NIC drivers to load..


----------



## Shane Daly

I was in RAID but the Intel option never appeared despite moving the jumper from LSI, now I'm going to dual boot W8 / OSX from the two separate drives instead I think.


----------



## singh_shady

I'd be interested in seeing a crystaldisk bench of your raid setup running on the marvel controller, if you would.
Glad it's up and running for you


----------



## jfrey

Hello,

A short update about my "blinking power led" for those whom may be interested: I replaced the MB for a new one and now all is working fine







With all the electronical stuff packaged inside, those little things are sensitive (it's been a long time since I got defective PC component!).


----------



## Shane Daly

Thanks, yes when I move forward a bit more I plan to speed test the 3 onboard RAID options for my C drive, I'll report back here.

The giant RAID runs of my Areca card FYI if that's what you meant.


----------



## Shane Daly

It took me 3 Z9PE-D8 WS boards including my current one before everything worked (the 2nd one was a refurb and practically DOA), and I still have to call tech tomorrow to see why the NIC shows up in Device Manager but will not be seen by the driver installer, so this 3rd board may yet be faulty.

ASUS have a terrific RMA setup and have been very helpful - a better solution would be much higher QC controls before shipping anything..??


----------



## singh_shady

Quote:


> Originally Posted by *Shane Daly*
> 
> Thanks, yes when I move forward a bit more I plan to speed test the 3 onboard RAID options for my C drive, I'll report back here.
> 
> The giant RAID runs of my Areca card FYI if that's what you meant.


That would be very informative and nice of you to do so.








Just jumped right in with the 'trusty' intel raid0 controller, with my 2 ssd's even though I have 3 ssd's in total to run, but didnt want to take the chance with the marvell. Would be brilliant to have the 3 ssd raid0 but not sure of the marvel controller.


----------



## YP5 Toronto

.....been tasked with building a DUAL CPU system. But everything I read up on various forums and on newegg point me to some crazy stories.

I have plenty of experience and work in the IT space.

Im looking at:

E5-2667 v2 x 2
GTX 780 x 3
plenty of ram
1500W PSU

question is... is the bios and everything mature enough? Why am I seeing so many issues?


----------



## DualFiz

Quote:


> Originally Posted by *YP5 Toronto*
> 
> .....been tasked with building a DUAL CPU system. But everything I read up on various forums and on newegg point me to some crazy stories.
> 
> I have plenty of experience and work in the IT space.
> 
> Im looking at:
> 
> E5-2667 v2 x 2
> GTX 780 x 3
> plenty of ram
> 1500W PSU
> 
> question is... is the bios and everything mature enough? Why am I seeing so many issues?


It's a quirky board... BUT, then again, so is every other motherboard made.

Oddly enough I have seen so many posts and messages from users of this board that make me ask myself "***? Why on Earth are they attempting such a complicated build with their complete lack of technical knowledge???"

Obviously, that's not the case for all of the issues... hell, I've seen some strange ticks using this motherboard that make it apparent there are hardware and or software (bios) issues left to resolve.

All I can do is speak from my experience, having built, owned and used several multi-cpu systems throughout my career... in that, this board is no different than the rest. AS LONG AS you have a solid understanding of the complexity regarding system building, AND do NOT cheap out on any components, you should be fine.

The one thing I've noticed throughout all my "internet research" with this motherboard is far too many builders load up on cheap cheap RAM. Meaning, they don't build with ECC RAM. I've noticed throughout that it seems those who use ECC, and ESPECIALLY ECC Buffered RAM do not seem to have the same struggles that many others have.

The other problem seems to come from ridiculous SLI gaming oriented video card configurations. This is not a good board for gaming, so, don't build one to game with.

Otherwise, go in careful, and use your intuition as a systems builder... you should be fine.

*knocks on wood*


----------



## TheBlademaster01

It advertises SLI so it should better support it







. DoA rate is high compared to let's say Supermicro boards. Also, ECC/reg can be slow for some people wanting performance out of the board (folding/other latency bound forms of DC).

It depends on the needs of the person in question whether they need SLI, CF, exotic RAID/NIC, Maximus or Xeon Phi setups. If Asus advertises this they are honestly to blame and not the customer for attempting it. It's a flaw of the board.

Other than that. If you have a properly functioning board (no DoA) and qualified parts you shouldn't have too many issues. Maybe some AIBs will have issues but latest BIOS is pretty mature AFAIK.


----------



## YP5 Toronto

Thank you for the in depth reply.


----------



## Shane Daly

ASUS are a joke and I am sorry I ever considered them.... My 3rd board now needs to be RMA'd - it will not see the 2nd CPU. I have switched CPU to check and fought through ASUS tech support phone hold ALL DAY SO FAR only to hear that all they can do is send out another board....

4 BOARDS!!!!

WHAT A JOKE OF A VENDOR.

BUYER BEWARE.


----------



## DualFiz

The post above is how I become confused... I own this board, I bought this board off eBay, I put two ES chips in it, 32gb of ECC/Reg ram, a Quardo K5000, some SSD's, HD's, two Dynatron R17s and a 860 watt Corsair PSU.

Turned it on, first time out of the gate, it booted, I installed windows, no problems, no complaints, no arguments.

When I look at my beautiful wife I realize I am most certainly a lucky man... but no way am I so lucky as to avoid the thousands of issues demonstrated in this thread by so many others.

So what's the difference here? Why do some have to go through a half dozen motherboards, and some of us boot first time, no problems? I just can't blame ASUS alone here, there have to be other factors at play.

PS: Shane Daly, I've got my fingers crossed the next board works for you, so many problems is undoubtedly frustrating. Have you considered a Supermicro or Tyan board? ...btw, I LOVE Tyan mobos. Would have bought one for this build, but got too good a deal to pass up on the Asus.


----------



## DualFiz

Quote:


> Originally Posted by *TheBlademaster01*
> 
> It advertises SLI so it should better support it
> 
> 
> 
> 
> 
> 
> 
> . DoA rate is high compared to let's say Supermicro boards. Also, ECC/reg can be slow for some people wanting performance out of the board (folding/other latency bound forms of DC).
> 
> It depends on the needs of the person in question whether they need SLI, CF, exotic RAID/NIC, Maximus or Xeon Phi setups. If Asus advertises this they are honestly to blame and not the customer for attempting it. It's a flaw of the board.
> 
> Other than that. If you have a properly functioning board (no DoA) and qualified parts you shouldn't have too many issues. Maybe some AIBs will have issues but latest BIOS is pretty mature AFAIK.


You do make a good point... in theory, as a "WS" grade motherboard, there should be no real compatibility issues in regards to most off the shelf hardware, but I do wonder if at times, we, as a community, are asking too much of this board, or expect too much from Asus? It's no excuse, but, in defense of Asus, they do not claim 100% compatibility with all configurations of hardware, just much of the hardware in and of itself.

It might just be, this motherboard can't handle such incredible demands?


----------



## TheBlademaster01

Much simpler system. I never had an issue either with my 470 and 2x ES CPUs but with my 780Ti things changed. I only have display via 1 DVI port. I've heard it may be due to having B0 Sandy chips so I'm going to try Ivy chips when I have the money.

I also got my board from eBay. Getting it out of the store (retail) might be risky.


----------



## YP5 Toronto

....I believe if they still had the SR-X in production, I would go that route.

I have always been an ASUS guy. 24+ odd machines for myself, friends and family...never a single problem.

But this board seems like its the black sheep of the family.... it doesn't invoke the same level of confidence that I am use to.

I will likely put this on hold. I may source something from the company I work for (Lenovo) and provide my buddy with something more focused on WORK. (his aspiration was a high end work station that he could game hard with).


----------



## Shane Daly

Quote:


> Originally Posted by *DualFiz*
> 
> The post above is how I become confused... I own this board, I bought this board off eBay, I put two ES chips in it, 32gb of ECC/Reg ram, a Quardo K5000, some SSD's, HD's, two Dynatron R17s and a 860 watt Corsair PSU.
> 
> Turned it on, first time out of the gate, it booted, I installed windows, no problems, no complaints, no arguments.
> 
> When I look at my beautiful wife I realize I am most certainly a lucky man... but no way am I so lucky as to avoid the thousands of issues demonstrated in this thread by so many others.
> 
> So what's the difference here? Why do some have to go through a half dozen motherboards, and some of us boot first time, no problems? I just can't blame ASUS alone here, there have to be other factors at play.
> 
> PS: Shane Daly, I've got my fingers crossed the next board works for you, so many problems is undoubtedly frustrating. Have you considered a Supermicro or Tyan board? ...btw, I LOVE Tyan mobos. Would have bought one for this build, but got too good a deal to pass up on the Asus.


Thanks DualFiz, this has become a nightmare start to my year - I'm 2 weeks behind my build schedule now and approaching deadlines.. I hear you that these are complex systems and we are expecting much, but we are also paying for equipment that is sold as capable.

I wish I had bought your board!!









I bought every single component with extreme care, every one is the flagship model for that part and matches the ASUS QVL.

The variable here is ASUS' QC - I do not believe that they send out good boards which have been checked, just the next sock avail whatever state it is in.

I have opened a fierce complaint and am demanding a guarantee that the 4th board has been bench tested in all aspects as 100% before they send it.

The definition of madness is repeating the same action but expecting a different outcome..... #furious


----------



## TheBlademaster01

I'm not sure SR-X is a lot better tbh.


----------



## DualFiz

Quote:


> Originally Posted by *TheBlademaster01*
> 
> Much simpler system. I never had an issue either with my 470 and 2x ES CPUs but with my 780Ti things changed. I only have display via 1 DVI port. I've heard it may be due to having B0 Sandy chips so I'm going to try Ivy chips when I have the money.
> 
> I also got my board from eBay. Getting it out of the store (retail) might be risky.


Hmm... that is one thing that also seems to come up a lot, video card compatibility. I had hoped the newer revisions of BIOS would resolve that?

Another thing we often overlook is how many of us are using ES CPU's, it'd be interesting to tally up those with ES, and those with production, and see if we can determine which configurations lead to greater instability.


----------



## DualFiz

Quote:


> Originally Posted by *YP5 Toronto*
> 
> ....I believe if they still had the SR-X in production, I would go that route.
> 
> I have always been an ASUS guy. 24+ odd machines for myself, friends and family...never a single problem.
> 
> But this board seems like its the black sheep of the family.... it doesn't invoke the same level of confidence that I am use to.
> 
> I will likely put this on hold. I may source something from the company I work for (Lenovo) and provide my buddy with something more focused on WORK. (his aspiration was a high end work station that he could game hard with).


I would most definitely not go SR-X... from everything I've read, that board is even "Worse" than the Asus board. So much so, it's next to impossible to even find for sale.

I'd still not give up on the Asus board, I ALMOST turned my back on it after reading this thread... but, I didn't, because I trusted Asus and I trusted my intuition. So, I bought it, and as I've said before, it worked perfectly for me right out of the box.

I'd rather suggest your buddy lay out the funds for two complete system builds... this build, for his workstation, and a second build, with a single 2011 CPU, to paired with a SLI 780ti or Titan for gaming. That way he's not trying to use either machine for something other than intended.

Worst case, build with this board, if it doesn't work, return it and exchange it for something else. Don't give up so quickly.


----------



## DualFiz

Quote:


> Originally Posted by *Shane Daly*
> 
> Thanks DualFiz, this has become a nightmare start to my year - I'm 2 weeks behind my build schedule now and approaching deadlines.. I hear you that these are complex systems and we are expecting much, but we are also paying for equipment that is sold as capable.
> 
> I wish I had bought your board!!
> 
> 
> 
> 
> 
> 
> 
> 
> 
> I bought every single component with extreme care, every one is the flagship model for that part and matches the ASUS QVL.
> 
> The variable here is ASUS' QC - I do not believe that they send out good boards which have been checked, just the next sock avail whatever state it is in.
> 
> I have opened a fierce complaint and am demanding a guarantee that the 4th board has been bench tested in all aspects as 100% before they send it.
> 
> The definition of madness is repeating the same action but expecting a different outcome..... #furious


No doubt, lost time is lost money... so I hope the next board resolves all of your issues!


----------



## ZDngrfld

The thing that bothers me about this motherboard is it's $500+. You'd expect if you're paying over $500 for a motherboard that it would have better QA. Every single Supermicro board I've ever owned or worked with has worked out of the box. The Z9, not so much.


----------



## DualFiz

Quote:


> Originally Posted by *ZDngrfld*
> 
> The thing that bothers me about this motherboard is it's $500+. You'd expect if you're paying over $500 for a motherboard that it would have better QA. Every single Supermicro board I've ever owned or worked with has worked out of the box. The Z9, not so much.


This is true... on the conversation of price, few motherboards are worthy of a $550 price tag. Regardless to quality.


----------



## Shane Daly

Now I am a little calmer it occurs to me that RMAing this 3rd board may be unnecessary, or at least a little premature - BOTH CPU have ALWAYS been present in the BIOS, live temp readings etc plus ALL RAM showing 100% good also.

As I have switched the Xeons it's clear both are sound, so the problem now is why does BIOS see both CPU and all hardware 100% but W8 will not? I am currently loading W7 on an identical SSD to isolate OS error.

Any wise heads, jump in here any time. If there's ever a Cinematography or film industry question, I'll owe you a deep and complete answer..









Thanks all

Shane


----------



## Shane Daly

BOTH CPU appear in Windows7, go figure...


----------



## ZDngrfld

What version of 8 are you using?


----------



## TheBlademaster01

Yeah you need at least Pro


----------



## turbotrk

Quote:


> Originally Posted by *TheBlademaster01*
> 
> Much simpler system. I never had an issue either with my 470 and 2x ES CPUs but with my 780Ti things changed. I only have display via 1 DVI port. I've heard it may be due to having B0 Sandy chips so I'm going to try Ivy chips when I have the money.
> 
> I also got my board from eBay. Getting it out of the store (retail) might be risky.


Quote:


> Originally Posted by *TheBlademaster01*
> 
> Much simpler system. I never had an issue either with my 470 and 2x ES CPUs but with my 780Ti things changed. I only have display via 1 DVI port. I've heard it may be due to having B0 Sandy chips so I'm going to try Ivy chips when I have the money.
> 
> I also got my board from eBay. Getting it out of the store (retail) might be risky.


Hi, I have the same problem, I 've just built a new system with z9pe, 2x xeon es2687w qa91 C1 and evga gtx780ti sc.
My graphic card is not recognized at all even with the latest bios flashed in. Also my second cpu if installed causes the system having to be reset several times before being able to post, then if I go to bios it says cpu2 not present. Iswapped the cpus and same result. With only 1 cpu it works fine altough no screen with gtx 780. I was told that es qa91 should be ok with Z9PE and the two cpus are identical, what do you think is the problem with it?


----------



## TheBlademaster01

The thing is QA91 are B0 Sandy Chips. They should not work with the latest BIOS though. What version are you running?

In any case. Flashing to 3506 makes the system post somewhat more consistently but it is still very buggy and pretty unstable. If you flash to 5004 and later it should not post with QAxx chips. I am now mining on my 780Ti and dual 2665s. It does so pretty well but each hour or so I get driver crashes and system lockups. But without touching it it seems to restore itself after a couple of minutes.

It's the only thing I can really do with the system at the moment though. Only my HDMI output works atm and only my TV is HDMI.

What you can do is:

Get Sandy C0/C1 chips (QBxx or if you can retail) and flash to the latest 5xxx BIOS (though 3506 should already fix this)

Get Ivy chips but flash to 5206 or later beforehand so the system will post with Ivy processors and non ECC memory

Get a BIOS chips with 0703 or older and use exclusively PCI-E 2.0 cards. .cap BIOSes wreck stability of B0 Sandy chips.

I'm going with option 2 if I can sell my chips and have the cash to upgrade.


----------



## turbotrk

The bios I have is 5304, with just cpu1 it works and posts, adding the second it does not post always at the first attempt and when it does it does not see cpu2 at all. So according what you say the problem comes from my QA91 cpu, if I replace it with a new ivy-bridge I should be able to boot with the gtx780 ti right? And of course the two cpus will be working togheter!
I will try if I can find a couple of xeons to try on my mb.

On my actual cpus I read this:

IMC10 (04)
Intel confidential
QA91 ES 3.06HZ
A4
C1 06F641 (04)


----------



## DualFiz

Do B0 Sandys not work with the newer BIOS revisions? I thought that was exclusive to a single BIOS release earlier last year that has since been corrected?


----------



## TheBlademaster01

Oh, not sure. I'm not willing to test it yet. I might have a system that may be unable to post









But turbotrk running that BIOS version makes me think that it might actually work.


----------



## turbotrk

I took the computer to the shop last evening, they will try the cpus on another Z9PE and also other cpus on my mainbord to understand if the second cpu is not recognized due to a faulty mb or it is the cpus not compatible. I have recommended the tecnician the try some ecc rams but he said that it doesn't make any difference because if one cpu boots it means the the ram is ok. I am not to shure about that but we will see.


----------



## DualFiz

Quote:


> Originally Posted by *TheBlademaster01*
> 
> Oh, not sure. I'm not willing to test it yet. I might have a system that may be unable to post
> 
> 
> 
> 
> 
> 
> 
> 
> 
> But turbotrk running that BIOS version makes me think that it might actually work.












Understood... this is why I've been hesitant to update my BIOS, even though I'm using v2 Xeons. I figure it's working for me now, why mess it up?









Quote:


> Originally Posted by *turbotrk*
> 
> I took the computer to the shop last evening, they will try the cpus on another Z9PE and also other cpus on my mainbord to understand if the second cpu is not recognized due to a faulty mb or it is the cpus not compatible. I have recommended the tecnician the try some ecc rams but he said that it doesn't make any difference because if one cpu boots it means the the ram is ok. I am not to shure about that but we will see.


Hmm... I'm pretty sure ECC and or ECC Registered in certain circumstances is required for dual CPU operation. BUT, I can't swear to that.


----------



## Shane Daly

Quote:


> Originally Posted by *DualFiz*
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> Understood... this is why I've been hesitant to update my BIOS, even though I'm using v2 Xeons. I figure it's working for me now, why mess it up?
> 
> 
> 
> 
> 
> 
> 
> 
> Hmm... I'm pretty sure ECC and or ECC Registered in certain circumstances is required for dual CPU operation. BUT, I can't swear to that.


Asus tech support yesterday mentioned a known issue with these boards, v2 Xeons and NON ECC RAM - check it out to be sure this isn't your problem?


----------



## Shane Daly

Quote:


> Originally Posted by *DualFiz*
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> Understood... this is why I've been hesitant to update my BIOS, even though I'm using v2 Xeons. I figure it's working for me now, why mess it up?
> 
> 
> 
> 
> 
> 
> 
> 
> Hmm... I'm pretty sure ECC and or ECC Registered in certain circumstances is required for dual CPU operation. BUT, I can't swear to that.


In W8 I only had CPU1 avail but all RAM, now in W7 I have both CPU but apparently only RAM sticks 1 & 2 are working (CPU-Z info) which cannot be the case if CPU2 is working.

Running tests and updates now, will report back.


----------



## Bsantos

Hi
I was looking around this thread of the forum and couldnt find this info.
I was on ebay and looked at two Xeon e5-2687w c0 stepping QB7R and i was thinking on buying them and use them with the Asus Z9PE-D8. I was afraid of buying them for the fact of being es cpu's.
I have some Quetions: 1.Will i run in with any issue with the bios ?
2.Do they have the same performance as the release one?
3. Will any feature be missing?
4. Will i see any decrease in performance in casual gaming ?
5.Are they usable toguether as I heard that some steppings wont work?

Thanks


----------



## DualFiz

Quote:


> Originally Posted by *Bsantos*
> 
> Hi
> I was looking around this thread of the forum and couldnt find this info.
> I was on ebay and looked at two Xeon e5-2687w c0 stepping QB7R and i was thinking on buying them and use them with the Asus Z9PE-D8. I was afraid of buying them for the fact of being es cpu's.
> I have some Quetions: 1.Will i run in with any issue with the bios ?
> 2.Do they have the same performance as the release one?
> 3. Will any feature be missing?
> 4. Will i see any decrease in performance in casual gaming ?
> 5.Are they usable toguether as I heard that some steppings wont work?
> 
> Thanks


From what little I know... C0 stepping e5 v1 Xeons are "QS" production CPUs. Translation = nearly retail. Rarely are there changes in the process once a product goes from QS to retail production. QS is not a typical engineering sample, it's a quality sample and typically the type sent to magazines and websites for review.

Compatibility should not be a problem as several websites which reviewed this motherboard at release were using C0 QS chips. However, you might need to "downgrade" your BIOS to achieve stability.

Performance, stability, features... should be nearly if not exactly all the same as retail.

BTW, what defines casual gaming to you? I do not recommend this board for gaming.


----------



## Bsantos

After work playing some video games maybe put a gtx 780 ti would that work ?


----------



## DualFiz

Quote:


> Originally Posted by *Bsantos*
> 
> After work playing some video games maybe put a gtx 780 ti would that work ?


Oh certainly, it will work... if gaming is truly a secondary function of your build, then it's not a bad idea. If gaming is your main objective, you can do so much (in fact, a LOT) better for a ton less money.









The 780ti matched to a nice Xeon will play games just fine, although, you might find driver issues and or motherboard/bios quirks get in the way from maximum performance.


----------



## Bsantos

i Think i read somewhere that this xeons at release had c1 and c2 steppings.
Will the bios from the motherboard interfier with upgradability like in the future a raid card or new gen gpu's ?
As i have limited time for gaming I wanted to try to do some youtube vidios of gaming and tutorials also livestream at the same time thats why I dont really want to go with something like an i7 extreme


----------



## DualFiz

Quote:


> Originally Posted by *Bsantos*
> 
> i Think i read somewhere that this xeons at release had c1 and c2 steppings.
> Will the bios from the motherboard interfier with upgradability like in the future a raid card or new gen gpu's ?
> As i have limited time for gaming I wanted to try to do some youtube vidios of gaming and tutorials also livestream at the same time thats why I dont really want to go with something like an i7 extreme


Probably... retail always gets a separate designation, however, that doesn't always mean there are functional differences with the fabrication.

As for the BIOS, it is certainly possible that at some point (Asus may already have) a release will disable compatibility with the C0 stepping CPUs. That is the risk you take with ES CPU's. Sure you save a ton of money, but down the line it might cost you a lot more to correct.

Basically, all you can do if you desire ES chips is build a system that is compatible with todays hardware and software... acknowledging the possibility that in the future you might not be able to upgrade any or all of the hardware in your system without the purchase of newer "retail" CPU's.


----------



## Bsantos

Ok I did some research around and there was one page that shows the model and it is a match as the problem with the virtualization can you be a little more specific does it still work ?


----------



## DualFiz

Quote:


> Originally Posted by *Bsantos*
> 
> Ok I did some research around and there was one page that shows the model and it is a match as the problem with the virtualization can you be a little more specific does it still work ?


From everything I've read, some C0, even C1 revision ES CPU's demonstrate a problem with VT-d virtualization... which appears to be a problem many, MANY Intel ES CPU's suffer from.

They still work fine for virtualization in general, just not using the VT-d enhancements. Which, not being an expert on virtualization myself, doesn't seem all that important anyways. Assuming you follow proper security methodology, I don't see how isolation of data flow directly to client would provide much benefit.

If you're building this system for a corporate environment, and will be providing high level secure connectivity options, then you shouldn't be buying ES chips from God knows who anyways.


----------



## Bsantos

ok Thank you if you can find any info on people having problems pls let me know thanks for the help
BTW the system is going to be at home thanks again


----------



## TheBlademaster01

Quote:


> Originally Posted by *DualFiz*
> 
> From everything I've read, some C0, even C1 revision ES CPU's demonstrate a problem with VT-d virtualization... which appears to be a problem many, MANY Intel ES CPU's suffer from.
> 
> They still work fine for virtualization in general, just not using the VT-d enhancements. Which, not being an expert on virtualization myself, doesn't seem all that important anyways. Assuming you follow proper security methodology, I don't see how isolation of data flow directly to client would provide much benefit.
> 
> If you're building this system for a corporate environment, and will be providing high level secure connectivity options, then you shouldn't be buying ES chips from God knows who anyways.


Probably when trying to use a high bandwidth NIC (10Gb/s) or SSD RAID array via a VM. VT-d allows the VM to connect directly to the hardware rather than having the kernel think about it -- which causes CPU overhead.

There are some benefits to VT-x though, like moving a VM across different physical machines without disconnecting them. I think VT-d is just needed for specific high performance applications. Other than that VT-x seems more flexible.


----------



## Shane Daly

DiskSpeedTest1.png 347k .png file


The first speed tests on my RAID.


----------



## czmiho

Hi, can please someone confirm, that it is possible to boot this MB with pre-installed BIOS using 2687W V2 and ECC memory at least to the point wehre i can flash new BIOS?

Does S3 sleep work well?

Thanks.


----------



## TheBlademaster01

Depends on which BIOS the board ships with. You need at least version 5004 but I don't think most will come shipped with that. You can always buy a pre-flashed BIOS chip from ebay though.


----------



## lowfat

Quote:


> Originally Posted by *TheBlademaster01*
> 
> I'm not sure SR-X is a lot better tbh.


Is avoid evga as well. I had two SR2 systems. Both of which had to be rma'd. After all the sr2 horror stories I've witnessed and heard id never buy one of their 2p boards.


----------



## othersfollow

I read a lot on this forum before I built my workstation, but I never wavered before picking this board. Mostly because I've had nothing but the best from Asus in the past.

I want to say, I finished my build last night and I have had no issues yet. I had to clear out the power once for a chassis interrupt error, but after that, I smooth booted right into Windows 8.1. Board is solid as can be so far and everything is running fast, cleanly and without any errors.

I mostly wanted to post because I read of mostly people having trouble with this, and I'm not experiencing the same problems at all. I don't want other people to be necessarily deterred in buying it. Here's what's in it:


Silverstone TJ09 Black
Asus Z9PE-D8 WS
Dual Intel 2.8 Xeon E5-2680 v2 Ivy Bridge EP
Noctua NH D14 SE2011 Workstation Cooler
Silverstone ST1200-G Evolution PSU
nVidia Quadro 4000
Crucial DDR3 PC3-14900 - 64GB
Crucial M500 SATA III SSD HD 960GB
ASUS PCE-AC66 AC Wireless Adaptor
Noctua NF-S12A Cooling Fans


----------



## Trotador

Quote:


> Originally Posted by *othersfollow*
> 
> I read a lot on this forum before I built my workstation, but I never wavered before picking this board. Mostly because I've had nothing but the best from Asus in the past.
> 
> I want to say, I finished my build last night and I have had no issues yet. I had to clear out the power once for a chassis interrupt error, but after that, I smooth booted right into Windows 8.1. Board is solid as can be so far and everything is running fast, cleanly and without any errors.
> 
> I mostly wanted to post because I read of mostly people having trouble with this, and I'm not experiencing the same problems at all. I don't want other people to be necessarily deterred in buying it. Here's what's in it:
> 
> 
> Silverstone TJ09 Black
> Asus Z9PE-D8 WS
> Dual Intel 2.8 Xeon E5-2680 v2 Ivy Bridge EP
> Noctua NH D14 SE2011 Workstation Cooler
> Silverstone ST1200-G Evolution PSU
> nVidia Quadro 4000
> Crucial DDR3 PC3-14900 - 64GB
> Crucial M500 SATA III SSD HD 960GB
> ASUS PCE-AC66 AC Wireless Adaptor
> Noctua NF-S12A Cooling Fans


what BIOS? retail or ES processors? What Crucial memory exactly?

thanks


----------



## othersfollow

Bios is 5206 - Came with the rev 1.04 board.

retail - i always go retail.

two of crucial's CT2K16G3ERSDD4186D - 32GB Kit 16GBX2 DDR3 DDR3-1866/PC3-14900 Reg ECC.

this way i can go up to 128 someday.


----------



## DualFiz

NICE build!!!

I swear Reg/ECC is the secret to this boards stability... I've read hundreds of users experiences with this board and while certainly not always the case, more often than not, when someone experiences issue, they're using non-reg non-ecc ram.

To me it demonstrates a poor mentality when developing a build around this motherboard, speed for gaming, or "bang for buck" is the wrong way to go, it has to be about highest quality products and stability over speed. It's a number cruncher, not a Crysis 3 dream machine.


----------



## othersfollow

Yeah, i render in 3d, so my builds are usually pretty simple and are heavy on asus boards and noctua fans

The noctuas were holding everything under 56C yesterday. That was the max i saw on any of the 20 cores at load.

I have to run some more tests, tweak the bios, and clean install windows 8.1 on the drive. Then run it for a while before i start crunching it on any renderings, but it' seems positive so far!

oh, and the board came from Newegg in case anyone is getting Ivy bridge-ep's and is worried about the onboard bios number.


----------



## TheBlademaster01

Quote:


> Originally Posted by *DualFiz*
> 
> NICE build!!!
> 
> I swear Reg/ECC is the secret to this boards stability... I've read hundreds of users experiences with this board and while certainly not always the case, more often than not, when someone experiences issue, they're using non-reg non-ecc ram.
> 
> To me it demonstrates a poor mentality when developing a build around this motherboard, speed for gaming, or "bang for buck" is the wrong way to go, it has to be about highest quality products and stability over speed. It's a number cruncher, not a Crysis 3 dream machine.


Or just people who do not want to waste a cycle each time on buffering when the correction or stability isn't needed but rather an obstruction when going for bandwidth







. Not only gaming profits from this btw. I don't even game, I'm also doing 3D and occasional distributed computing and ECC severely limits performance in everything not 3D. But I think aside from the occasional DoAs the issue is also certain ES chips. Intel and Kepler had issues with forcing PCI-E 3.0 even on the HEDT platforms.

Then there is the thing about people running SM boards like X9DAi or X9DA7 without RAM, vid card, RAID or ES chips problems. Something tells me the guys who write firmware drop the ball quite often over at Asus. Especially seeing as version 3109 broke manual bClk settings completely.


----------



## TheBlademaster01

But it's no surprise Registered RAM is softer on the IMC due to lower fan out seen by the IMC (which is also why it supports higher density modules).


----------



## RickSFO

Blademaster (and other experts!),

I also have B0 chips (2687W) and am having stability issues. My video card is PCI-E 2.1 (AMD Firepro 4900) and the RAM is on the ASUS approved vendor list (GSKILL F3-12800CL9Q-16GBZL). I have the timing set to 9-9-9-24 which is what's on the sticks and on the asus approved vendor list. I have one Samsung 830 Series SSD. And an LG GH24 DVD writer. Nothing else.

On BIOS 0703 the system would lock up occasionally. Maybe every 5 hours of use. Sometimes it was when working hard, but sometimes when just browsing the net. It seemed random. When I say lock up, sometimes the screen would just freeze. Sometimes it would go black. Sometimes, but not often, the system would try to reboot itself, but got stuck. A power cycle would allow it to boot.

Then on the ASUS support page I saw the latest BIOS 5304 with this note "Improve compatibility for add-on VGAs while 2 CPUs installed". So I thought great, they fixed the stability issue! So I upgraded to 3109 and the system seemed ok. Then I went to 5304 and all hell broke loose.







The system wouldn't boot. It goes into a weird loop where the power would kick on, the fans would spin up for a second, then stop, and repeat this cycle forever. If I pull the battery out and reset the CMOS then it at least brings up a screen that said press F1 to go into bios and "recover BIOS settings". If I do that, and hit F5 to "load optimized defaults" then the system boots. But it only sees one CPU and half the RAM. I tried putting the memory timing back to what I mentioned above but no luck.

ANy ideas? Is it the non-ECC RAM? Is BIOS 5304 not compatible with the B0 chips? The guy I bought the system from said it was stable and he was doing long rendering runs. I did install the video card, SSD, and DVD writer. Could those be related?

At this point I'm going to have to order a BIOS chip with 0703 installed. Or should I try something between 0703 and 5304? Or I could try ECC memory. Or both...

Thanks in advance!

Rick


----------



## TheBlademaster01

I still have not resolved my issues either. It should be the B0 chips that are incompatible with .cap BIOSes. I'm using BIOS 3506 and it boots pretty consistently but I only have display via HDMI do it's still pretty useless.

BIOS 0503/0703 and a PCI-E 2.0 card would be your best bet. Or C0/C1 chips (or Ivy) if you want to use newer graphics cards.


----------



## shoek

@othersfollow

Is that nVidia Quadro card a PCI-E 3.0 card? That seems to be another area where people are getting into trouble with this mobo.


----------



## TheBlademaster01

No, Quadro 4000 = GTX 460 equivalent. It's PCI-E 2.0 Fermi. I had no issues running the 470 at .rom bioses. Quadro K series and later will have issues.


----------



## othersfollow

Yep, just as blademaster said, the quadro 4000 a PCI-E 2.0. I'm waiting to upgrade that until nVidia makes PCI-E 3 quadro cards. The K6000 is the only one now, and I'm not doubling the price of my system to use it.

I'm hoping there's a M4000 coming out soon.


----------



## TheBlademaster01

K5000 etc. are PCI-e 3.0 as well but I take it that you want GK110.


----------



## shoek

@all

Can anyone summarize the video card situation with this mobo? Has there been success running modern PCI-E 3.0 GPU's (GTX770/780 for instance) in SLI configuration?
I ready back 20 pages or so in the thread and heard that the PCI-E 3.0 cards may cause BCLK overclocking to max out at 104-106 (similar to SuperMicro HyperSpeed mobos).


----------



## othersfollow

Nope, only the K6000. K5000 and lower are PCI-e 2.0:

http://www.nvidia.com/object/quadro-k5000.html#pdpContent=2


----------



## Shane Daly

Quote:


> Originally Posted by *Bsantos*
> 
> ok Thank you if you can find any info on people having problems pls let me know thanks for the help
> BTW the system is going to be at home thanks again


I thought I had solved all my mobo issues but it seems not! I have posted in this thread with previous board issues, 3 RMA cycles and this board seemed good with Windows 7, with W8 it would not show CPU2.

The OS shows 128Gb RAM (Kingston QVL 1600) but I am experiencing some issues with card combinations which pointed me to explore the RAM further - CPU-Z shows only slots 1 & 2 so I think the system is running on 32gb right now?

BIOS is 5304, GTX Titan, dual 12core Xeons.

I am stuck! I can't send another board back, and the fact that the BIOS sees all the memory tells me it's not a hardware issue but hopefully an OS issue?

Thanks for any helps guys.

Shane


----------



## TheBlademaster01

I don't see how that is possible tbh since GTX 680 is basically a K5000 and is PCI-e 3.0. Same chip, different firmware. I think it's a typo on their website. Also, K5000 is one of the cards that have issues with B0 chips as well.

Pretty sure I saw someone running it at 3.0 speeds as well in hwinfo64

Here's PNY data-sheet.

http://www.pny.eu/data/sitedynamic/File/Quadro/Datasheets/QuadroK5000_by_PNY_Datasheet.pdf


----------



## DualFiz

I run a Quadro K5000 in my Asus workstation and have had no problems with it so far *knocks on wood*

Regarding the interface technology. Nvidia states in their specifications that the K5000 is a PCIe 2.0 bus card. However, PNY claims their K5000 cards are PCIe 3.0.

BUT, as we all know, all Quadro cards are reference cards. Nvidia does not allow custom components to be used and remain a "Quadro".

What does that tell us? Either PNY knows something Nvidia doesn't? Or PNY is lying.

I suspect PNY to be lying.

REGARDLESS, if you read up on video cards and the bandwidth provided via PCIe 2.0 and 3.0 you'll find consistently, unless dealing with SLI or Crossfire situations, the differences in performance are anywhere from 0.5% to 3%.

Translation = there is no difference.

So, don't sweat it. PCIe 3.0 isn't worth saving or waiting for.


----------



## othersfollow

Yeah, I'm not sure. Newegg has it at 2.0 as does the other sites I've seen. And on PNY's site, it's listed in the specs as 2.0:

http://www3.pny.com/NVIDIA-Quadro-K5000-font-colordc0431Newfont-P3224C365.aspx

Either way, I'm holding out for a Maxwell based quadro. They've got to be around the corner.


----------



## RickSFO

Thanks Blademaster.

My video card is PCI-E 2.1 Is that a potential problem?

At one point (when I was still on BIOS 0703) I took the video card out completely and used the motherboard VGA. No difference... Still locked up on occasion.

How about my non-ECC RAM. Is that an issue?

Rick


----------



## Shane Daly

Quote:


> Originally Posted by *RickSFO*
> 
> How about my non-ECC RAM. Is that an issue?
> 
> Everything I have read so far points to ECC Reg. RAM being the only way to go, anything seems to cause problems.
> 
> Rick


----------



## TheBlademaster01

I don't think it should be a problem. You had a V4900 right? That is an HD 6670 equivalent. I'm pretty sure the CP.u should support it. My 780 Ti wouldn't even post on bios revisions older than 3109.

The lock-ups are strange though even without VGA. Do you get BSODs? Did you check the Q-Code (onboard LED)?

I've been using my chips with Dominator Platinums no problems. It really started when installing the 780Ti in my case.


----------



## RickSFO

My RAM is on the approved list for this board. However, I assume ASUS tested it with retail chips.

So is the ECC RAM only needed for the ES chips (mine are B0)?? Why would that be?


----------



## TheBlademaster01

I have B0 chips as well.

ECC RAM was needed to have Ivy chips post on BIOS revision 5004. That was the first official release to support Ivy processors. And breaks support with Sandy B0 chips. I heard of someone running Sandy B0 on 5206 but I'm not man enough to try it


----------



## RickSFO

Never seen a true BSOD from this board. Video either freezes with whatever was on the screen at the time, or goes black, or rarely, tries to reboot itself.

Unfortunately I don't recall the error code. It happened so many times I stopped looking at it. I'm ordering a BIOS chip with 0703 so I'll be able to repeat the scenario.


----------



## RickSFO

In the current state, with BIOS 5304, why doesn't it see the 2nd CPU??? Again they are 2687W B0.

My video card is PCI-E 2.1 (AMD Firepro 4900) and the RAM is on the ASUS approved vendor list (GSKILL F3-12800CL9Q-16GBZL). I have the timing set to 9-9-9-24 which is what's on the sticks and on the asus approved vendor list. I have one Samsung 830 Series SSD. And an LG GH24 DVD writer. Nothing else.


----------



## TheBlademaster01

I'd run the RAM at default settings. So use the timings and frequency assigned by the motherboard. What you're saying sounds a lot like the issues I have with my 780Ti at the moment. But again, that is due to the fact that these chips don't really work well with .CAP BIOSes. Even my 470 craps out like this. With 0503 I had no issues whatsoever.

My RAM is Dominator Platinum 1866MHz. It functions correctly at factory settings except for the fact that the board doesn't really like 1866MHz so I can only run 1600MHz reliably.


----------



## Shane Daly

Are you on Windows 7 or 8? On W8 mine wouldn't see CPU2 either, solved by loading Windows 7. Unfortunately that meant it now doesn't see RAM slots 3-8 - any ideas anyone??!


----------



## RickSFO

W7 64


----------



## RickSFO

With BIOS 5304, I'm using the "optimized defaults" by hitting F5 in BIOS. I haven't changed anything manually. That's the only way the system boots at all. But it still doesn't see the 2nd CPU.

Is there any way to downgrade the BIOS without ordering a new chip?


----------



## ZDngrfld

Quote:


> Originally Posted by *RickSFO*
> 
> With BIOS 5304, I'm using the "optimized defaults" by hitting F5 in BIOS. I haven't changed anything manually. That's the only way the system boots at all. But it still doesn't see the 2nd CPU.
> 
> Is there any way to downgrade the BIOS without ordering a new chip?


Look on Asus's support page for the Z9. I think there's a method listed on there. I think you'll be able to go as far back as 3109


----------



## TheBlademaster01

Quote:


> Originally Posted by *Shane Daly*
> 
> Are you on Windows 7 or 8? On W8 mine wouldn't see CPU2 either, solved by loading Windows 7. Unfortunately that meant it now doesn't see RAM slots 3-8 - any ideas anyone??!


Which versions of W7 and W8 were you running?

Quote:


> Originally Posted by *RickSFO*
> 
> With BIOS 5304, I'm using the "optimized defaults" by hitting F5 in BIOS. I haven't changed anything manually. That's the only way the system boots at all. But it still doesn't see the 2nd CPU.
> 
> Is there any way to downgrade the BIOS without ordering a new chip?


No, I'm afraid not. You can only downgrade as far as version 3206. It will throw an error message with "Incorrect file type. Make sure file has .CAP" if you try to flash to 3109 and 0703 and 0503 will just throw "Outdated BIOS file" Something along those lines. I don't remember the exact lines but basically this is what it will say.

As soon as you flash to .CAP bios it won't go back to .ROM bioses.


----------



## Shane Daly

Quote:


> Originally Posted by *TheBlademaster01*
> 
> Which versions of W7 and W8 were you running?
> 
> Both were 64 Pro.
> 
> No, I'm afraid not. You can only downgrade as far as version 3206. It will throw an error message with "Incorrect file type. Make sure file has .CAP" if you try to flash to 3109 and 0703 and 0503 will just throw "Outdated BIOS file" Something along those lines. I don't remember the exact lines but basically this is what it will say.
> 
> As soon as you flash to .CAP bios it won't go back to .ROM bioses.


----------



## TheBlademaster01

Oh okay, that's weird then... Yet another reason not to upgrade to Win 8


----------



## DualFiz

RickSFO... what is the brand name / wattage of the PSU you're using in your build?


----------



## RickSFO

Antec 1200 High Current Pro


----------



## DualFiz

Quote:


> Originally Posted by *RickSFO*
> 
> Antec 1200 High Current Pro


So you're using an adapter for the second 12v 8 pin EPS connection?

Have you tried moving that adapter to a clean chain? ...with no other devices connected to any of the remaining molex connections?

...assuming it's not already like that.

Another thing I've sometimes wondered about in regards to this motherboard is PSU's with dual EPS connections vs. those without. My PSU has dual EPS built in. I wonder if it makes any difference in stability?


----------



## Donkey1514

Quote:


> Originally Posted by *DualFiz*
> 
> NICE build!!!
> 
> I swear Reg/ECC is the secret to this boards stability... I've read hundreds of users experiences with this board and while certainly not always the case, more often than not, when someone experiences issue, they're using non-reg non-ecc ram.
> 
> To me it demonstrates a poor mentality when developing a build around this motherboard, speed for gaming, or "bang for buck" is the wrong way to go, it has to be about highest quality products and stability over speed. It's a number cruncher, not a Crysis 3 dream machine.


no..... After 4 boards, 3 different memory kits and every stepping of ES chips, I'm quite certain most users are having problems because they own either B1/B0 V1 chips + early, unstable bios that doesn't fully support newer gpu's or they are trying to run V2 chips with an early bios that doesn't support V2 chips. ECC vs non-ECC has zero affect on the board working based on my testing.


----------



## phillydee

Haven't been on here in a while. System has been running stable except for one thing: I constantly have been getting a "System Interrupt" in my task manager and I've nailed it down to my Sound Blaster audio card. I've changed PCI slots so that the card is on a separate PCI bus color (blue/grey... video card on blue, sound card on grey etc) and that seemed to help but after a given period of time, I get the system interrupt again. It's irritating because I use audio quite often in my work and having a dropout or glitch happen from every several seconds to a few minutes gets really annoying.

So.... question: does anyone use a PCI sound card and which one works well? I'm considering using an Asus Xonar sound card instead of the Sound Blaster.

Basic details on rig:

Win8.1 pro (OEM)
dual E5-2665 C0 ES (or C1 stepping... can't remember)
32gigs DDR3 1600 ECC
Corsair AX1200 PSU
GTX 680

I'm running Bios 5004
NO onboard audio drivers installed or seen on the OS

Soundcard: SoundBlaster Recon 3D PCI card

Any input is appreciated... thanks!!


----------



## aurik94

I recently ordered one of these off of newegg -- and I'm already regretting it. I have the following components:

Asus Z9PE-D8
Corsair AX860 PSU
Corsair Carbide Air 540 case
2x Xeon 2620v2
8x Kingston KVR16R11S4/4 4G DDR1600 ECC Registered DRAM (total 32G)
PNY GeForce GTX760 2GB video card
2x Dynatron R17 coolers
2x Samsung EVO 840 256G SSD
Asus MIO-892 audio

So I get the system all configured and power it on... nothing. System halts at 0xBA on the LED (memory test). I strip it down to one CPU and 4 DIMMS - POSTs fine. I then put the second CPU and find that it will boot with 1 or 2 DIMMs, but not 4 -- hence slot F1 is dead.

I tried updating the BIOS to the latest revision, and that didn't help anything, either.

Dammit. Now I've got to wait for the whole RMA process, and I'm 30 minutes away from the nearest UPS facility.


----------



## othersfollow

can't you return it to newegg within 30 days?


----------



## TheBlademaster01

I've heard of that issue before, sorry man. DoAs really suck


----------



## aurik94

I'll call newegg and check it out. On their invoice they say to call ASUS tech support, but DoA may be an exception...


----------



## DualFiz

Have you tried moving the RAM around? In other words, swapping all the sticks into different slots?


----------



## aurik94

I've tried 3 different sticks in that slot. The problem is definitely the slot, not the sticks.


----------



## lowfat

Quote:


> Originally Posted by *DualFiz*
> 
> I swear Reg/ECC is the secret to this boards stability... I've read hundreds of users experiences with this board and while certainly not always the case, more often than not, when someone experiences issue, they're using non-reg non-ecc ram.


I can't agree. I have had the same issues w/ registered ECC as I have w/ regular DDR3.


----------



## Donkey1514

Quote:


> Originally Posted by *Donkey1514*
> 
> no..... After 4 boards, 3 different memory kits and every stepping of ES chips, I'm quite certain most users are having problems because they own either B1/B0 V1 chips + early, unstable bios that doesn't fully support newer gpu's or they are trying to run V2 chips with an early bios that doesn't support V2 chips. ECC vs non-ECC has zero affect on the board working based on my testing.


Quote:


> Originally Posted by *lowfat*
> 
> I can't agree. I have had the same issues w/ registered ECC as I have w/ regular DDR3.


----------



## t_ski

Hello. Another new Z9PE-D8 user here having trouble. I got mine from Newegg last Wednesday, and I'm having trouble getting it to POST. Well, it did post once, and it said that it could not recognize the keyboard. I figured it did not like the USB keyboard I had on a KVM, so I tried hooking a PS/2 keyboard up straight. Ever since then, it turns on with the Power and Reset buttons lit up, along with the green LED in the bottom right corner. The LAN jack even flashes with the activity lights. But when I push the power button the fans move slightly and that's it. System is as follows:

two E5-2680v2 ES chips (tried each one by themselves in socket 1)
two sticks of Samsung green ram, 4GB each (tried one stick at a time, both sticks and swapped them. Also tried one 4GB stick of Samsung ECC in A1)
XFX Radeon 6570 video card (tried in every slot, and tried with the onboard enabled & disabled both. Also tried an unknown chipset Nvidia card I have laying around - older card that Ubuntu wouldn't identify)
Antec EarthWatts 650W PSU with EPS adapter on a clean run (PSU is rated for 34A and each of the 12V rails, which is more on one of those than the Extreme PSU calculator says the entire system needs. I also tried a Cooler Master 650W PSU.)
Silverstone AR02 coolers (made sure they were not too tight by trying to twist the heatsink as I tightened the screws)
One Seagate 160GB HHD (also tried a Samsung 80GB HDD and no drive connected)
One DVD burner (and tried without that one connected, too)

Everything is sitting open on top of the box the board came in. I can't even get into the system to see what BIOS is on it, and I get absolutely no codes on the debug LED. The only two things I can think left to try (at least out of what I have available to work with here) is to put in both CPUs and test to see if I get any other kind of response out of the system, or possibly hook it up to the PSU on my current system, which is a Corsair AX1200.

Does anyone else have any suggestions before I try to RMA? I admit I've only made it through about 20 pages of the thread so far, but I'll keep reading in the meantime. Thanks


----------



## t_ski

Well, I tried both CPUs and it didn't work (I really didn't expect it to be any different). Then I took the cables from my AX1200 and hooked them up to the board. I pressed the power button and too my surprise I saw both CPU fans spinning!!! Then I heard crackling, saw smoke, then a small fireball appeared under the heatpipe on the board.









Going to have to RMA for sure. Not sure if I can RMA through Newegg now or if it will have to go through Asus.


----------



## anubis1127

Woah, that sounds rather exciting and frustrating at the same time.


----------



## Marchitect

Quote:


> Originally Posted by *phillydee*
> 
> Haven't been on here in a while. System has been running stable except for one thing: I constantly have been getting a "System Interrupt" in my task manager and I've nailed it down to my Sound Blaster audio card. I've changed PCI slots so that the card is on a separate PCI bus color (blue/grey... video card on blue, sound card on grey etc) and that seemed to help but after a given period of time, I get the system interrupt again. It's irritating because I use audio quite often in my work and having a dropout or glitch happen from every several seconds to a few minutes gets really annoying.
> 
> So.... question: does anyone use a PCI sound card and which one works well? I'm considering using an Asus Xonar sound card instead of the Sound Blaster.
> 
> Basic details on rig:
> 
> Win8.1 pro (OEM)
> dual E5-2665 C0 ES (or C1 stepping... can't remember)
> 32gigs DDR3 1600 ECC
> Corsair AX1200 PSU
> GTX 680
> 
> I'm running Bios 5004
> NO onboard audio drivers installed or seen on the OS
> 
> Soundcard: SoundBlaster Recon 3D PCI card
> 
> Any input is appreciated... thanks!!


I am having issues with my system BSOD every few days. My research lead me to think the Realtek audio drivers may be the culprit. So I disabled the Realtek audio drivers and purchased a SoundBlaster X FI Titanium. I put it in the bottom most slot which is blue. The card works great but I still get the BSOD. My BSOD is related to CAP12 Event ID 513. Many posts about this say it is related to the Realtek audio. But so far I have tried a new sound card and still have the issue.
I'm going to try to update all my motherboard drivers and see if that helps. It's been about 10 months since I've updated the drivers. I am using the latest Bios.


----------



## YP5 Toronto

This won't go away....

The challenge in front of me is 2 x 2697v2 (not ES) + Two GTX 790s (when they are launched).

My buddy is insistent on making this materialize.

arg... all these posts scare me though.

I would likely go with ECC ram.


----------



## turbotrk

Hi, I took my pc to the shop a couple of weeks ago , I had problems with two 2687w ES QA91 that could not work paired and my gtx 780ti not being recognized at all. They tryied two ivy bridge v2 cpus and it worked ok and also posted with gtx 780ti. I was told that QA91 work with Z9PE but probabbly not togheter because it only posts in single cpu mode.


----------



## phillydee

After running LatencyMon it's saying that storahci.sys is causing the highest DPC routine execution time.

I have all the latest chipset drivers installed.... this is kind of nuts. What's weird too is if I do any hardware changes, its fine for about a week, and then it crops up.


----------



## t_ski

Newegg is going to RMA the board for me, so I don't have to go through Asus. However, when I pulled the chips off the board to send it off, I noticed that there seems to be some damage to the pads underneath it. I'm not sure what these pads are responsible for, but I'm leery of trying to put it in a socket again.


----------



## DualFiz

Quote:


> Originally Posted by *t_ski*
> 
> Newegg is going to RMA the board for me, so I don't have to go through Asus. However, when I pulled the chips off the board to send it off, I noticed that there seems to be some damage to the pads underneath it. I'm not sure what these pads are responsible for, but I'm leery of trying to put it in a socket again.


Do you mean where the CPU interfaces to the pins on the motherboard? They are actually called pads... well, technically I believe. I've heard references made to them as "landing pads" as well.

What sort of damage are we talking about? Scorch marks? Scratches? Are any of them "cut"?

My assumption is the damage you've experienced with your motherboard was due to a short created by the CPU pins themselves running power to ground.

Damaged pads would most certainly cause that.

It might very well be that nothing was wrong with the motherboard at all and in fact, all that is wrong with your configuration is the CPU's themselves.

BUT, it's equally possible that when the motherboard "shorted" out and burst into flames there was damage caused to the CPU(s). Will they work in another motherboard? Hard to say... it could very well be they're now broken, or that they will now cause damage in future motherboards too.


----------



## t_ski

Quote:


> Originally Posted by *DualFiz*
> 
> Do you mean where the CPU interfaces to the pins on the motherboard? They are actually called pads... well, technically I believe. I've heard references made to them as "landing pads" as well.
> 
> What sort of damage are we talking about? Scorch marks? Scratches? Are any of them "cut"?


Here's a pic:



One pad is definitely half missing (the bottom one I marked), and the other one looks slightly managed. I've been looking online trying to find a whitepaper or something else that would state what each of these pins do, but haven't found anything yet.
Quote:


> Will they work in another motherboard? Hard to say... it could very well be they're now broken, or that they will now cause damage in future motherboards too.


I haven't tested either of these is a working board, as the only other one I have is my P9X79 Pro, and I need that 100% this week. When I get the replacement board, I might test these on that board first. I too am worried that it would damage another board.


----------



## DualFiz

Quote:


> Originally Posted by *t_ski*
> 
> Here's a pic:
> 
> 
> 
> One pad is definitely half missing (the bottom one I marked), and the other one looks slightly managed. I've been looking online trying to find a whitepaper or something else that would state what each of these pins do, but haven't found anything yet.
> I haven't tested either of these is a working board, as the only other one I have is my P9X79 Pro, and I need that 100% this week. When I get the replacement board, I might test these on that board first. I too am worried that it would damage another board.


Odd... is there physical damage? In other words, if you run your finger across them, does it feel as if material is missing? It almost looks like they've been dug out.

It doesn't look from the photo like there are pads which would short to each other in any way, unless one of the pins on the motherboard was crossing two pin pads.

Hopefully they're just ground pins, in fact, the majority of pins/pads on a CPU are just grounds. So, if you lose a few, no big deal.

It might be in your best interest to go to a Frys or Microcenter (whatever local large chain computer store that you might live near) and buy a single socket 2011 motherboard. Get it home and try each chip on their own to see if they boot alright. When you're done (yes, I recognize some might struggle with this idea morally), just return the motherboard for a full refund. (assuming the CPU doesn't blow the motherboard up or something)

Keep us updated... and most importantly, I'm sorry to hear of your struggles, it sounds terribly frustrating.


----------



## gurmukh23

Hey,

After reading all the horror stories about this Mobo here, am seriously thinking about getting the Gigabyte GA-7PESH3 instead. Looks pretty cool. What are your thoughts?


----------



## gurmukh23

Here is the link
http://b2b.gigabyte.com/products/product-page.aspx?pid=4468#sp


----------



## singh_shady

Quote:


> Originally Posted by *gurmukh23*
> 
> Hey,
> 
> After reading all the horror stories about this Mobo here, am seriously thinking about getting the Gigabyte GA-7PESH3 instead. Looks pretty cool. What are your thoughts?


I believe the board to be solid and being a dual socket board there are some things that new users have to consider making before the move.
I had the problem of only one CPU working until I swapped the socket the CPUs were sitting in and it booted right up.
Then some bios settings needed to be changed, if your an enthusiast and know how to troubleshoot issues, the board is great. It's not for beginners. Asus have a great express RMA service for workstation boards too.

Also, if your not into any sort of video, 3d model rendering type work. It has no real benefit. Multi threaded applications is where this board shines, since the CPU's can't be overclocked.


----------



## ICEscream

Hi guys,

just bought the items below to start building my new workstation. It has been years since I did my last build. Do you think
these components are ok?

Asus Z9PE-D8 WS
2- Xeon 2630 v2
32gb dd3 ram (8x4gb) 1866mhz
Gtx 780
Corsair RM-1000 80 Plus Gold Power Supply
2x Thermaltake NiC F4 CPU Cooler

After reading some of the posts, i am now a bit nervous if I could make this work.


----------



## gurmukh23

Quote:


> Originally Posted by *singh_shady*
> 
> I believe the board to be solid and being a dual socket board there are some things that new users have to consider making before the move.
> I had the problem of only one CPU working until I swapped the socket the CPUs were sitting in and it booted right up.
> Then some bios settings needed to be changed, if your an enthusiast and know how to troubleshoot issues, the board is great. It's not for beginners. Asus have a great express RMA service for workstation boards too.
> 
> Also, if your not into any sort of video, 3d model rendering type work. It has no real benefit. Multi threaded applications is where this board shines, since the CPU's can't be overclocked.


Thanks although I am familiar with this board and its issues, I was just asking if anyone had experience using the Gigabyte one. Haven't seen much info on it. I will be using iray so am doing a multi GPU build with dual 2697 v2 and awaiting the new Maxwell cards (4 GPUs) on the new Corsair 1500 PSU in the 900D


----------



## singh_shady

My apologies. Assumed way ahead of myself.

You are going all out with the cpu and graphics power. Awesome spec.








To be honest, I never knew of the Gigabyte board until you mentioned it, possibly because I hadn't come across any users or reviews. If its anything like their mainstream boards, it will be reliable.

If you end up going with it, I'd very much like to read your experiences and thoughts on it. I know people on this thread would find it particularly valuable.


----------



## t_ski

Quote:


> Originally Posted by *DualFiz*
> 
> Odd... is there physical damage? In other words, if you run your finger across them, does it feel as if material is missing? It almost looks like they've been dug out.
> 
> It doesn't look from the photo like there are pads which would short to each other in any way, unless one of the pins on the motherboard was crossing two pin pads.
> 
> Hopefully they're just ground pins, in fact, the majority of pins/pads on a CPU are just grounds. So, if you lose a few, no big deal.
> 
> It might be in your best interest to go to a Frys or Microcenter (whatever local large chain computer store that you might live near) and buy a single socket 2011 motherboard. Get it home and try each chip on their own to see if they boot alright. When you're done (yes, I recognize some might struggle with this idea morally), just return the motherboard for a full refund. (assuming the CPU doesn't blow the motherboard up or something)
> 
> Keep us updated... and most importantly, I'm sorry to hear of your struggles, it sounds terribly frustrating.


Yes, some of the pad is actually missing. I can feel it on the top pad with less damage, but the bottom one seems smooth. Unfortunately, I have no retailers anywhere around here to get a board from, and I'm still waiting for newEgg to receive the RMA (much less send out the replacement).
Quote:


> Originally Posted by *gurmukh23*
> 
> Hey,
> 
> After reading all the horror stories about this Mobo here, am seriously thinking about getting the Gigabyte GA-7PESH3 instead. Looks pretty cool. What are your thoughts?


I believe I looked at that board before settling on the Asus. I was put off by the lack of support for Windows 8.X.


----------



## aurik94

Quote:


> Originally Posted by *aurik94*
> 
> I'll call newegg and check it out. On their invoice they say to call ASUS tech support, but DoA may be an exception...


So an update: Newegg confirmed the board as DOA and is sending out a new one. Hopefully it'll be here by the weekend.


----------



## anubis1127

Quote:


> Originally Posted by *aurik94*
> 
> So an update: Newegg confirmed the board as DOA and is sending out a new one. Hopefully it'll be here by the weekend.


Good luck! Hopefully the new board is flawless.


----------



## ignoramus

UPGRADES!!

I bought the pair of 2687w's that were on ebay for a reasonable price! SSD to come within the next week or so..

The CPUs are QA92s which i thought were B0. Does anyone know anything about these?

Also, im a little confused. Im running BIOS version 3302 which is .CAP and i was under the impressions that .CAP does not work with B0 stepping CPUs, however both this pair and my previous pair (2665s B0) run smoothly. What is the latest BIOS i can upgrade to? Do i risk going any higher?

NB my startup from cold is a little rocky. it powers down before it powers itself back up and displays the Asus boot screen about 5 times before starting windows..

As is:
100% load GHz: 3.224(across all cores)
Highest turbo: 3.432
64GB ECC RAM

Cinebench: 24.07
Benchwell: 1088.86


----------



## TheBlademaster01

Well, I'm running B0 chips as well on version 3506 and it posts but display through GTX 780 Ti is intermittent and I have weird issues like system lockups and RAM only being enabled in triple channel.

What GPU do you have?


----------



## ignoramus

An old GTX 460. I dont think its pcie 3.0.. pretty sure its 2.0. More info below:


And how would i know if im running in quad channel?

Also, HWinfo64 causes bluescreens when it opens so i dont want to run it. It did the same on my previous 2665s.

Might your lockups have anything to do with 2P? Some programs would crash unless i set the processor affinity to the cores on one processor.


----------



## anubis1127

Yeah, Fermi cards are all PCIe 2.0.


----------



## TheBlademaster01

No, it isn't 2P that causes the lockdowns. I never had these issues with .ROM bioses (0703 and before). It shouldn't BSOD though, but yeah HWinfo reports triple channel.

Maybe your system works correctly because you only used PCI-E 2.0 cards. My GTX 470 also causes issues now though.


----------



## ignoramus

Strange. CPUz reports quad channel for me. Downgrade to 3302? Any reason to be on such a high version?

Are you on the same 2665s as before? I remember we had the same CPUs.

edit - at which bios version did the B0s lose their support?


----------



## TheBlademaster01

I have tried all the BIOSes until 3506. 3506 is the most reliable. Well actually 0503 was, but I can't downgrade to ROM BIOS anymore.

And yes, I still have those 2665s 

E:

B0 lose suppport with version 5004. I've read about people running version 5206 on them but I'm not that tough to try it


----------



## adamburnett82

Quote:


> Well, I'm running B0 chips as well on version 3506 and it posts but display through GTX 780 Ti is intermittent and I have weird issues like system lockups and RAM only being enabled in triple channel.


Have you solved your 780 Ti issue? Is it just a BIOS problem with the 3506 BIOS?

I want to put a pair of them in my rig.

Here's what I'm building in my workstation, which I want to GPU render in Vray RT and maybe Iray.

(2) - E5-2620 v2
32 GB Kingston ECC Valueram
Corsair RM 1000 power supply
Samsung 256 SSD boot drive
(4) - Seagate Barracuda 1TB storage drives
Fractal Design XL R2 (modded to accept the form factor)

I haven't purchased the CPUs, RAM or GPUs yet, but I'm pretty set on the CPUs. I could change the GPUs, but the 780 Ti's are what I was hoping for.

Also, what BIOS would you recommend?


----------



## anubis1127

Quote:


> Originally Posted by *adamburnett82*
> 
> Have you solved your 780 Ti issue? Is it just a BIOS problem with the 3506 BIOS?
> 
> I want to put a pair of them in my rig.
> 
> Here's what I'm building in my workstation, which I want to GPU render in Vray RT and maybe Iray.
> 
> (2) - E5-2620 v2
> 32 GB Kingston ECC Valueram
> Corsair RM 1000 power supply
> Samsung 256 SSD boot drive
> (4) - Seagate Barracuda 1TB storage drives
> Fractal Design XL R2 (modded to accept the form factor)
> 
> I haven't purchased the CPUs, RAM or GPUs yet, but I'm pretty set on the CPUs. I could change the GPUs, but the 780 Ti's are what I was hoping for.
> 
> Also, what BIOS would you recommend?


With the proposed CPUs, you would want the latest BIOS, and a 780 Ti should work swimmingly, or any PCIe 3.0 GPU for that matter.

If you mean the Define XL R2, the z9pe-d8 WS fits, you could mod it for all the mobo standoffs, but you don't need to.


----------



## adamburnett82

Awesome! Thanks for the info, Also good to know that I don't have to tap any holes...I may grind off the threads of a couple of stand offs and still put them on, we'll see.


----------



## ZDngrfld

Or just get some of these plastic standoff things


----------



## TheBlademaster01

Quote:


> Originally Posted by *ignoramus*
> 
> Strange. CPUz reports quad channel for me. Downgrade to 3302? Any reason to be on such a high version?
> 
> Are you on the same 2665s as before? I remember we had the same CPUs.
> 
> edit - at which bios version did the B0s lose their support?


I need to thank you. BIOS 3302 enables Quad Channel for me. Not sure about stability but this already gives me a lot more performance


----------



## ignoramus

Awesome, glad it helped! Although you are now running PCIe v2.0 :\ I wonder what is causing your instability. My 2665 B0 system was rock solid.

Some notes on my last system that maybe you can try?
-BCLK no higher than 104
-Graphics card in the 5th slot down (blue)
-My raid0 HDDs (spinning platter) plugged in to the two right most white sata sockets


----------



## singh_shady

Sadly im having to downgrade from my Z9PE D8 setup for the time being. If anyone has any interest check it out here


----------



## ABnormAL77

hi all

i just updated my bios to Version 5304 and now my pc will not post it cames up wit q code 62 ive to 2 e5 2680 es

what can i do to get it to boot again?


----------



## othersfollow

This board is really a beast. I just scored a 1435.93 on Benchwell with it (wither.org), and I have far from the best Xeon chips:
http://www.maxwellrender.com/index.php/benchwell/id/2507


----------



## ABnormAL77

this board is the worst and ill never go ASUS again, i cant run 4 way sli cant over clock and now update bios and it will not boot with the ES xeon in it

so if some ones looking into getting this ASUS z9pe-de ws DONT


----------



## othersfollow

i think it really depends on what you're doing with it.

i have the board in a pretty basic setup - no frills. the gear i selected was top of the line, but i'm not running a complex config, like 4 way sli or a raid, or even a watercooling system. I've got one quadro in it. i've got one ssd. and i'm using the tried and true noctua air coolers.

i'm running 5304 as well, but not having the troubles that others are reporting. i'm sorry to hear you're having troubles - hope you can figure them out.


----------



## ABnormAL77

i spent to much money on this pc

2x xeon e5-2680 ES 6c/12t 3.0ghz (show up in windows 32 cores)
asus z9pe-d8 ws
lain li pc-d8000 case
1500w ps
2x120g ssd for windows
64gig ram 8x8g
16x2gig hdd
6x750g hdd
24 port rain card pcix8 v3 with 4g cache
lots of low speed fans
one evga 690 gtx
win 7 ult 64

i was going to make it a new game computer but i cant get my 4x680 gtx to detect all the time in windows the mother board wasn't all was given enough resource to have all 4 to work and cant OC the cpu or ram so had to make it into a new server for me

i just ordered a bios chip off ebay with a old Version 3206 i know that works

and one more thing i hate about it too i cant change the boot sequence so it take a long time to load









never get another ASUS board again (all was get a motherboard with more then one bios chip)

so i had to use my old system again

EVGA sr-2
2x xeon x5690 OC 3.47ghz to 4.6ghz 2x6core plus 12t so 24 cores in windows
48g ram 12x2g OC
4x evga 680 SC
4xssd reain 0 for win 7 64
2x1500w PS
blu-ray burner
still good 3dmark im in the top 99%


----------



## ABnormAL77

oh if i get a XBOX ONE or a PS4 they will be the slow computer in my house


----------



## TheBlademaster01

Quote:


> Originally Posted by *ABnormAL77*
> 
> hi all
> 
> i just updated my bios to Version 5304 and now my pc will not post it cames up wit q code 62 ive to 2 e5 2680 es
> 
> what can i do to get it to boot again?


Quote:


> Originally Posted by *ABnormAL77*
> 
> i spent to much money on this pc
> 
> 2x xeon e5-2680 ES 6c/12t 3.0ghz (show up in windows 32 cores)
> asus z9pe-d8 ws
> lain li pc-d8000 case
> 1500w ps
> 2x120g ssd for windows
> 64gig ram 8x8g
> 16x2gig hdd
> 6x750g hdd
> 24 port rain card pcix8 v3 with 4g cache
> lots of low speed fans
> one evga 690 gtx
> win 7 ult 64
> 
> i was going to make it a new game computer but i cant get my 4x680 gtx to detect all the time in windows the mother board wasn't all was given enough resource to have all 4 to work and cant OC the cpu or ram so had to make it into a new server for me
> 
> i just ordered a bios chip off ebay with a old Version 3206 i know that works
> 
> and one more thing i hate about it too i cant change the boot sequence so it take a long time to load
> 
> 
> 
> 
> 
> 
> 
> 
> 
> never get another ASUS board again (all was get a motherboard with more then one bios chip)
> 
> so i had to use my old system again
> 
> EVGA sr-2
> 2x xeon x5690 OC 3.47ghz to 4.6ghz 2x6core plus 12t so 24 cores in windows
> 48g ram 12x2g OC
> 4x evga 680 SC
> 4xssd reain 0 for win 7 64
> 2x1500w PS
> blu-ray burner
> still good 3dmark im in the top 99%


I get occasional 62 (or b2?) codes on my board as well. But that's because of my chips. What is the SPEC number on the 2680s? (QXXX)


----------



## 82ghost82

Hi all!
I recently upgraded the cpus of my workstation, from 2x xeon e5-2630 (retail) to 2x xeon e5-2687w (QS cpus, QB7R C0 stepping).

This is my system:
MB: Asus z9-ped8 ws (obviously)
Video card: Nvidia quadro 600 (slot 1)
Power supply: Cooler master 850W
RAM: Samsung ecc memory, 64 gb (8x8gb) ddr3, 1333 Mhz
HD: 1x 250 gb SSD, 1x 1 tb 7200 rpm
Coolers: 2x Corsair h80i

The bios version of the mobo is actually 3302; the system boots fine, everything seems to work good.
I would like to manually change ram clock speed, from auto (1333 Mhz) to 1600 Mhz; with the "old" cpus I tried and that worked just fine, but now the system doesn't boot into windows (if I return back to 1333 Mhz it boots again).

I would like to upgrade the bios version of the mobo, but it could be that with the newest version the system won't boot due to QS cpus; anybody knows what is the latest working bios version to boot with these QS cpus?

I found a user using v. 5004 with QB7R and he only had to change the slot of the video card.


----------



## TheBlademaster01

QB7R should work with all the releases. Only Sandy B0 chips have issues with 5004. Oh and Ivy chips don't work with 5004 without ECC RAM (and of course older BIOSes like 3506 since those don't have Ivy support).


----------



## 82ghost82

thank you theblademaster01!
I successfully updated the bios to v.5304 and all is working well.
I was able also to clock the ecc ram to 1600 Mhz.
Now I'm happy









EDIT:
Just another thing about bios v.3302: in the main tab cpu clock speeds showed up at 1200 Mhz; now they're showing up at 3100 Mhz which is correct.
So it's safe to update to 5304 with QB7R!


----------



## TheBlademaster01

Nice to hear man









Those QB7R are beasts


----------



## ABnormAL77

just ordered some biso chips off ebay , and now i have to wait for them







so i cant use my pc until then
i dont know what i had so i ask for Version 3206, 3302, 3506 and 5206 hope one of this will work ,
around$100 all up







it sucks


----------



## 82ghost82

Quite expensive...from Germany to Usa it should be around 70 USD (shipping included) for 4 chips.


----------



## eyepiz

Hello all!

I have recently jump on the Dual CPU band wagon!. I work in rendering and animation using 3DS Max and V-Ray, also do video compositing in aftereffects.

I have my set up and running and have some question about some issues I have encountered.

Workstation Specs

Asus Z9PE-D8 WS (Bios 5304)
(2) Intel Xeon 8 Core CPU's E5-2680v2 ES (QE5F)
(32) Gigs of G.Skill Trident X 2400 running @ 1600
(4) Evga Titan GTX in SLI
(1) OCZ Vertex 4 SSD (OS)
(1) Samsung Evo 840 (Data)
Enermax Max Revo 1500W PSU
Windows 7 Pro

First Problem I encountered.

After successfully running the memory at 1600 I tried to run the memory at 1866. everything booted up ran AIDA64 for 24 hours with no problem, then suddenly my Samsung drive suddenly became unresponsive and extremely slow I thought the drive had gone bad. I had to use robocopy to copy all the data to another drive that took almost an entire day.

I dropped the memory back down to 1600 just incase and now the drive seems fine, weird, because all the other drives were fine it was just the Samsung running bad. both drive are running on the Intel SATA 3 controller.

Does this board not run well at 1866 or does it require ECC for 1866?

Second Problem I encountered.

The graphic card test (4) Titans.

I Ran 3DS Max with V-Ray RT and because it does not use SLI it fired up and ran all four video cards with no problem.

The second test.

I enabled SLI and ran 3DMark 11 It ran the test well, but scores were extremely slow in 4-way SLI. I think only two cards are being used, when I ran a single card test the score was on par with a single card, but not the 4-way SLI configuration.

Does this board not support 4-Way SLI? had anyone had a 4-way SLI configuration running successfully?

I really appreciate any suggestions.

Thanks you!

Eric


----------



## TheBlademaster01

1866MHz is difficult but some chips are able to do it. My system doesn't POST at speeds above 1600MHz.

4-Way SLI should be supported but did you plug in the 4-pin molex connector on the board and measure GPU usage?


----------



## othersfollow

im using 1866 ran here but R, ECC. it's worked fine so far.


----------



## eyepiz

Quote:


> Originally Posted by *othersfollow*
> 
> im using 1866 ran here but R, ECC. it's worked fine so far.


Cool...what models are you running?


----------



## eyepiz

Quote:


> Originally Posted by *TheBlademaster01*
> 
> 1866MHz is difficult but some chips are able to do it. My system doesn't POST at speeds above 1600MHz.
> 
> 4-Way SLI should be supported but did you plug in the 4-pin molex connector on the board and measure GPU usage?


Nope...I was wondering what that molex was for...I had an ASUS rampage formula 4 never used the molex and ran 4-way SLI fine.

strange because V-Ray runs all the cards at 100 load but its a cuda application and does not make use oft he SLI bridge.

I will give that a shot.

Thanks!


----------



## othersfollow

crucial:
http://www.crucial.com/store/partspecs.aspx?imodule=CT2K16G3ERSDD4186D


----------



## VitOne

Hello!

I would like to know if I can safely run 2 QA87 Xeon ES CPUs (2,3Ghz, 16Cores) on this motherboard.
I would also like to know if an Enermax 85+ 1250W will work for it (it has 2 8PIN connectors: http://www.enermax.com/userfiles/image/revolution85_chart12.jpg).

The CPUs are step B0 so I am a bit scared about BIOS updates. Thanks!


----------



## TheBlademaster01

Only get those if you have bios revision 0703 or older and use PCI-E 2.0 GPUs.


----------



## VitOne

Thanks for you answer.
Unfortunately I need to run Windows 8.1 on the system and I have a K6000 VGA...
I am waiting to assembly the machine and already got all the components.
So maybe I should directly send back everything or there is an hope to have make the components work?


----------



## ZDngrfld

Quote:


> Originally Posted by *TheBlademaster01*
> 
> Only get those if you have bios revision 0703 or older and use PCI-E 2.0 GPUs.


+1
Find some C0s or C1s SB-EP chips or some of the new IVB-EP chips. The amount of people who start complaining because their ES chips don't work the way they want bothers me... You should know you're going to have quirks when you're buying non-retail...


----------



## VitOne

Thanks.
Any place where I could find a C0 or C1 CPU with 8 cores? Also in PM.
Do you think that it is worth to give a try to the B0 CPUs and the Quadro K6000 or not?
Unfortunately I have them and I already paid them.


----------



## TheBlademaster01

Quote:


> Originally Posted by *VitOne*
> 
> Thanks for you answer.
> Unfortunately I need to run Windows 8.1 on the system and I have a K6000 VGA...
> I am waiting to assembly the machine and already got all the components.
> So maybe I should directly send back everything or there is an hope to have make the components work?


Well, don't flash the BIOS to 5004. You need 3109 or later for Win 8 support btw.

And well, I'm running a 780 Ti with B0 chips and it is a lot of headaches really. Strangest bugs and instability. I wouldn't recommend running B0 chips with a K6000 if you even get it to post. I must say that Ubuntu works a lot better on my system, though it still has some bugs now and then. Also it names my 780 Ti Ashley which makes it not recognized in some applications and forces my RAM in triple channel.

Weirdest things really. I would follow ZDngr's advice really.


----------



## VitOne

Quote:


> Originally Posted by *TheBlademaster01*
> 
> Well, don't flash the BIOS to 5004. You need 3109 or later for Win 8 support btw.
> 
> And well, I'm running a 780 Ti with B0 chips and it is a lot of headaches really. Strangest bugs and instability. I wouldn't recommend running B0 chips with a K6000 if you even get it to post. I must say that Ubuntu works a lot better on my system, though it still has some bugs now and then. Also it names my 780 Ti Ashley which makes it not recognized in some applications and forces my RAM in triple channel.
> 
> Weirdest things really. I would follow ZDngr's advice really.


Thanks. I'll try to find some C0/C1 CPUs!

Do you think that my PSU will work properly? Enermax 1250W 85+


----------



## TheBlademaster01

I think so. I've heard stories about some PSUs being incompatible with this board but you should be fine with 2x EPS I think.


----------



## ZDngrfld

Quote:


> Originally Posted by *VitOne*
> 
> Thanks. I'll try to find some C0/C1 CPUs!
> 
> Do you think that my PSU will work properly? Enermax 1250W 85+


I don't see that this PSU would be an issue. It's a bit dated, but seems like it should be up to the task.


----------



## VitOne

Thanks,
Could you please suggest me a good case for this motherboard? What ATX standard is it?


----------



## TheBlademaster01

SSI-EEB, comparable to E-ATX but the mounting holes don't fully align. Coolermaster Cosmos S, Silverstone Raven RV03, Lian Li V750, V2120. I don't really know, there are quite some.


----------



## ZDngrfld

Quote:


> Originally Posted by *VitOne*
> 
> Thanks,
> Could you please suggest me a good case for this motherboard? What ATX standard is it?


It's SSI-EEB. E-ATX cases will work. The only difference is that there's a couple standoff mounts that don't line up. You can buy a couple plastic standoffs to give yourself some more support. I have mine in an old Coolermaster Cosmos-S I bought years ago.


----------



## eyepiz

Quote:


> Originally Posted by *andrefsf*
> 
> Hi guys!
> 
> This baby is helping us running some CUDA enabled applications in Linux. Great motherboard! Overall the layout is very good, but the headers on the bottom (usb2.0, usb3.0 and serial) should be right angled. If you are using a 4 GPU configuration is no easy to connect the last GPU...
> 
> 
> 
> 
> Regards


Nice!...what CPU coolers are those? I'm using Dynatron R17's and temps are high 60's under full load. I was concerned about being able to go bigger with a GPU in the top slot.


----------



## ZDngrfld

Quote:


> Originally Posted by *eyepiz*
> 
> Nice!...what CPU coolers are those? I'm using Dynatron R17's and temps are high 60's under full load. I was concerned about being able to go bigger with a GPU in the top slot.


Those are Coolermaster Hyper-212 Evos.


----------



## eyepiz

Quote:


> Originally Posted by *ZDngrfld*
> 
> Those are Coolermaster Hyper-212 Evos.


How would those compare to the Dynatron R17's? worth the swap?


----------



## VitOne

Is the Xeon E5 V2 step B1 a good choice for this motherboard or I will also have problems with latest BIOS and PCI-Express 3.0 VGAs like the B0/B1 stepping of the E5 "V1" Xeon CPUs?
Thanks!


----------



## TheBlademaster01

I'm not aware of such issues. They don't exist at this point, but with ES you never know. B1 Ivy isn't like B0 Sandy though, it's a bit more polished. I believe even retail Sandy-E and Sandy-EP had issues with PCI-E 3.0 at some point because Intel couldn't test it at the time (there were no PCI-E 3.0 cards while taping out).


----------



## VitOne

Thanks! This is a very unhappy situation. What would be the "best buy" in your opinion? An E5 V2?


----------



## TheBlademaster01

Well C0/C1 Sandy are QS chips and should be barely retail. Ivy chips should of course be more efficient and have better memory controllers for higher speed RAM (and more performance if you get something like E5-2680V2s or faster).


----------



## VitOne

I could buy a couple of QB79 CPUs. Do you think that they will work? Unfortunately this are the best CPUs that I could buy know (about 450$ for each CPU, 8 Core, step C0).


----------



## VitOne

Otherwise for a similar price I could get a QE5F, 8 Core, Step L1 V2. I don't know if I can trust the L1 stepping... Any idea?


----------



## TheBlademaster01

Yes they should work fine. Those are QS E5-2680s. 3.1GHz all core turbo, just like retail.


----------



## VitOne

Quote:


> Originally Posted by *TheBlademaster01*
> 
> Yes they should work fine. Those are QS E5-2680s. 3.1GHz all core turbo, just like retail.


Thanks, I also found for similar prices some 10-Core L1 stepping - QDNR - and some 8-Core L1 stepping - QE5F (both V2). The 10 Cores seems very interesting, but I am a bit scared about the L1 stepping. What do you think?


----------



## VitOne

Quote:


> Originally Posted by *VitOne*
> 
> Thanks, I also found for similar prices some 10-Core L1 stepping - QDNR - and some 8-Core L1 stepping - QE5F (both V2). The 10 Cores seems very interesting, but I am a bit scared about the L1 stepping. What do you think?


I am sorry for all this posts. I have been told that the QB7B (C0, "V1") should work fine with PCI-Express 3.0 (a "colleague" owns it) at it is also cheaper and it could be a good one to give a try. What do you think? It should be an early version of the E5 2660.


----------



## TheBlademaster01

Quote:


> Originally Posted by *VitOne*
> 
> Thanks, I also found for similar prices some 10-Core L1 stepping - QDNR - and some 8-Core L1 stepping - QE5F (both V2). The 10 Cores seems very interesting, but I am a bit scared about the L1 stepping. What do you think?


ZDngrfld has QDNR IIRC. You should ask him about it, but I'm fairly sire that they should work without issues
Quote:


> Originally Posted by *VitOne*
> 
> I am sorry for all this posts. I have been told that the QB7B (C0, "V1") should work fine with PCI-Express 3.0 (a "colleague" owns it) at it is also cheaper and it could be a good one to give a try. What do you think? It should be an early version of the E5 2660.


QB7B should work as well as it is C0, it's just a but slower than QB79.


----------



## VitOne

I think that I will go with the QB7B. I will update to something with more cores in the next months if possible, now I hope that everything will work fine.


----------



## ZDngrfld

Quote:


> Originally Posted by *VitOne*
> 
> Thanks, I also found for similar prices some 10-Core L1 stepping - QDNR - and some 8-Core L1 stepping - QE5F (both V2). The 10 Cores seems very interesting, but I am a bit scared about the L1 stepping. What do you think?


I do have the QDNRs. They've been working fine since day one, although I'm not using them for my workstation or anything. I just have them in a Debian headless setup so my usage is far from what you'd be using them for.


----------



## eyepiz

Quote:


> Originally Posted by *TheBlademaster01*
> 
> Yes they should work fine. Those are QS E5-2680s. 3.1GHz all core turbo, just like retail.


Hi Blademaster!...I'm new to the dual cpu and just built a machine using a few ES CPU's









I have the a pair or E502680 V2 QE5F (8 Core) what stepping are these of how can you find out? I read this tread for several hours yesterday and did not find much info on them.

I did run into issues running at 1866 with these. It booted up fine ran some benchmarks well, however, after a few days the Intel sata controller did weird stuff like a one of my SSD seemed unresponsive and I first thought the drive died, but after bring it back down to 1600 the problem seems to be gone.

In not running ECC at the moment but plan to eventually.

Eric


----------



## TheBlademaster01

I generally search for these chips a lot via ebay and taobao and remember the (Qxxx) SPEC numbers.

Those would be L1 chips (I believe that's the same thing as B1). You also have L1 10 core variants of those. Those are called QD29.


----------



## andrefsf

eyepiz, if I remember correctly those are CoolerMaster's Hyper 212 EVO...


----------



## VitOne

Quote:


> Originally Posted by *ZDngrfld*
> 
> I do have the QDNRs. They've been working fine since day one, although I'm not using them for my workstation or anything. I just have them in a Debian headless setup so my usage is far from what you'd be using them for.


Thanks again for your answers.

I have a few more questions and hopefully somebody can help me:

1) Can I use 2 SSDs in RAID0 with TRIM enabled on this motherboard? I need them as primary boot for a Windows 8.1 system.

2) I have 4*8Gbyte of this RAM modules: http://www.crucial.com/store/partspecs.aspx?imodule=BLT8G3D1608DT1TX0 the manufacturer (Crucial) says that they are NOT compatible with our beloved motherboard. Do you think that I can give this RAMs a try or it is better to get something different? Could you suggest me decent performing RAMs? I know for sure that for best performance all 8 slots should be filled, so maybe I can go for 4Gbytes modules for a total of 32Gbytes of RAM (I don't need more at the moment).

3) I will watercool the CPUs and I would like to watercool also the motherboard. I know that I will loose one of the PCI-Express slots (I need only 2 at the moment) and I would like to know if anybody here experienced any problems/benefits using watercooling (I would go with the solution from EK).


----------



## Jann

Quote:


> Originally Posted by *VitOne*
> 
> I think that I will go with the QB7B. I will update to something with more cores in the next months if possible, now I hope that everything will work fine.


Hey, I have QB7B cpus, and they have been rock solid!
I can't possibly imagine why would you watercool them, except maybe for visuals. Two Hyper 212 Evo's are more than enough to keep them cool under full rendering load. And you can get smaller/silent coolers similar to these. Some of the hotter 2690/2687w chips might need more, but not these








I have used RAID before with two Samsung 830 128Gb ssd's both with Win7 and Win8. There was some hiccup with raid drivers I think, but it installed in the end.
Can't comment about memory, though I'm using 4 kits of KHX1600C10D3B1K2/16G which aren't supported either


----------



## VitOne

Thanks for your message.

I will watercool the CPUs because I already have the waterblocks and usually watercooling gives me less noise. Hopefully in the feature I will update this computer with 12 core high frequency CPUs and the watercooling will be more useful.

It would be great if you can suggest me the right RAID drivers. I am a bit scared abut the TRIM function, does anybody know anything about *TRIM support with RAID* on this motherboard?


----------



## shoek

Quote:


> Originally Posted by *VitOne*
> 
> It would be great if you can suggest me the right RAID drivers. I am a bit scared abut the TRIM function, does anybody know anything about *TRIM support with RAID* on this motherboard?


My understanding is that Intel's support of TRIM on RAID is only with newer chipsets than the C602/X79 that this board has. I've run RAID0 of 2-4 SSD's without TRIM on my systems for years with no performance issues. Modern SSD's have background garbage collection that functions effectively in the absence of TRIM.


----------



## VitOne

Quote:


> Originally Posted by *shoek*
> 
> My understanding is that Intel's support of TRIM on RAID is only with newer chipsets than the C602/X79 that this board has. I've run RAID0 of 2-4 SSD's without TRIM on my systems for years with no performance issues. Modern SSD's have background garbage collection that functions effectively in the absence of TRIM.


Thanks, I'll try to see if a RAID0 without TRIM can work for me.


----------



## VitOne

Quote:


> Originally Posted by *Jann*
> 
> Hey, I have QB7B cpus, and they have been rock solid!
> I can't possibly imagine why would you watercool them, except maybe for visuals. Two Hyper 212 Evo's are more than enough to keep them cool under full rendering load. And you can get smaller/silent coolers similar to these. Some of the hotter 2690/2687w chips might need more, but not these
> 
> 
> 
> 
> 
> 
> 
> 
> I have used RAID before with two Samsung 830 128Gb ssd's both with Win7 and Win8. There was some hiccup with raid drivers I think, but it installed in the end.
> Can't comment about memory, though I'm using 4 kits of KHX1600C10D3B1K2/16G which aren't supported either


Could you please tell me if the StepSpeed is working on your CPUs? Did you set anything particular to make them work?
I was not able to see the QB7B SteepSpeed here: http://www.cpu-world.com/sspec/QB/QB7B.html and I am a bit worried because I do need a CPU capable of running a 3Ghz on a single core when I am using single software.
Thanks!


----------



## Jann

Quote:


> Originally Posted by *VitOne*
> 
> Could you please tell me if the StepSpeed is working on your CPUs? Did you set anything particular to make them work?
> I was not able to see the QB7B SteepSpeed here: http://www.cpu-world.com/sspec/QB/QB7B.html and I am a bit worried because I do need a CPU capable of running a 3Ghz on a single core when I am using single software.
> Thanks!


It's marked here http://www.cpu-world.com/CPUs/Xeon/Intel-Xeon%20E5-2660.html
SpeedStep works ok for me. CPU's idle at 1.2GHz, and go from 2.7GHZ(all cores) to 3GHz(1 or 2 can't remember) when loaded.
If you like, I can run some specific apps and screen cpu-z or whichever software you need








Next monday/tuesday, as I'll be away for this weekend.


----------



## VitOne

Quote:


> Originally Posted by *Jann*
> 
> It's marked here http://www.cpu-world.com/CPUs/Xeon/Intel-Xeon%20E5-2660.html
> SpeedStep works ok for me. CPU's idle at 1.2GHz, and go from 2.7GHZ(all cores) to 3GHz(1 or 2 can't remember) when loaded.
> If you like, I can run some specific apps and screen cpu-z or whichever software you need
> 
> 
> 
> 
> 
> 
> 
> 
> Next monday/tuesday, as I'll be away for this weekend.


Thanks, if you have time, I'd like to see SolidWorks (usually runs on 1/2 cores while modelling, more for FEA, all for PhotoView 360) and other software like Modo.


----------



## ICEscream

Hi Guys,

I just want to share my newly assembled system. All working rock solid (bios 5103).
I am using this machine for 3d renderings and animation.

*Asus z9ped8
*2 x Xeon E5 2630 v2 (BCLK 102)
*2 x Nic F4 (under 40C under normal use)
*32 gb quad channel 1866 ddr3 non ecc (8x4gb geil)
*GTX 780
*Samsung 840 evo 512 gb
*Corsair RM 1000
*Corsair 650D (modified the internals) - motherboard just barely fits the case, removed couple of standoffs and put some plastic screws on the mounting holes of the motherboard ( I think i only got 4 screws to align with the standoffs). I could have picked a bigger case but I will be moving in couple of months and I was looking at the smallest case possible with a sturdy build.
*Bunch of old hdds


----------



## TheBlademaster01

Very nice


----------



## squidman

New E7-2800 v2 processors...15 cores...

Looking at Serve the Home (http://www.servethehome.com/haswell-ep-ivy-bridge-ex-ivy-bridge-ep-sandy-bridge-ep/) it's still not clear to me whether the new chips (specifically the 15-core dual-socket ones) will be compatible with the z9pe-d8/ws....

Anyone figure this out? 2x15 cores is a lot more than 2x8 for rendering....!







I'm aware that there are 12-cores that will work - e5-26xx v2 (provided I assume you install ecc-ram), but hey, losing out on an extra 6-cores. Of course hoping ES versions will be floating about (not really about to drop 12 grand on 2 processors)....

My 2x e5-2665 B0 are still doing fine, but any increase in rendering speed is welcomed...


----------



## shoek

re: E7 processors... my understanding is that they use a version of Socket 2011 that is not pin-compatible with the E5 Xeon v1/v2 that the Z9PE-D8 uses.
I'm not really sure what mobos will support the E7's -- probably something high end from Supermicro and the like

The Haswell EP Xeon processor should be out this fall using yet another (3rd) version of Socket 2011, also incompatible with the Z9PE-D8. I'm hoping Asus has a workstation dual processor motherboard for the Haswell Xeon that uses a more modern chipset than the Z9PE-D8's C602/X79 chipset.


----------



## eyepiz

Hi all!...I wanted to share my build, so far running rock solid.









It running high 60's under load, let me know if I should change out the CPU coolers.

Thanks!

Workstation Specs

Asus Z9PE-D8 WS (Bios 5304)
(2) Intel Xeon 8 Core CPU's E5-2680v2 ES (QE5F)
(32) Gigs of G.Skill Trident X 2400 running @ 1600
(4) Evga Titan GTX in SLI
(1) OCZ Vertex 4 SSD (OS)
(1) Samsung Evo 840 (Data)
Enermax Max Revo 1500W PSU
Windows 8.1 Pro


----------



## YP5 Toronto

Quote:


> Originally Posted by *eyepiz*
> 
> Hi all!...I wanted to share my build, so far running rock solid.
> 
> 
> 
> 
> 
> 
> 
> 
> 
> It running high 60's under load, let me know if I should change out the CPU coolers.
> 
> Thanks!
> 
> Workstation Specs
> 
> Asus Z9PE-D8 WS (Bios 5304)
> (2) Intel Xeon 8 Core CPU's E5-2680v2 ES (QE5F)
> (32) Gigs of G.Skill Trident X 2400 running @ 1600
> (4) Evga Titan GTX in SLI
> (1) OCZ Vertex 4 SSD (OS)
> (1) Samsung Evo 840 (Data)
> Enermax Max Revo 1500W PSU
> Windows 8.1 Pro


This setup goes against all the posts of people have trouble with sli, non ecc memory, ect.

Damn..this build is sexy.


----------



## YP5 Toronto

Have you run any benchmarks? cinebench, 3dmark, valley? pls post.


----------



## eyepiz

Thanks!...

The memory didn't seem to run well above 1600mhz so I settled on 1600mhz...SLI works fine. I built this rig mainly for rendering with 3DS Max and Vray / Vray RT, the 16 core/32 trerads just rip through rendering!.

All four Titan card fire up like champ and crunch through Cuda apps like Vray RT with no problem at full speed, but 3D benchmarks such as 3DMark 11 they scale terrible like they are only coded to run a single socket or something.

3DMark score 1 4-Way SLI
3DMark score 2 Single Titan
3DMark score 3 (X79 Board running 4-Way SLI for comparison).


----------



## eyepiz

Quote:


> Originally Posted by *YP5 Toronto*
> 
> Have you run any benchmarks? cinebench, 3dmark, valley? pls post.


Just ran valley.

It looks like GPU clocks don't boost to full speeds in SLI...weird anyone else running SLI NVidia cards?


----------



## TheBlademaster01

Quote:


> Originally Posted by *squidman*
> 
> New E7-2800 v2 processors...15 cores...
> 
> Looking at Serve the Home (http://www.servethehome.com/haswell-ep-ivy-bridge-ex-ivy-bridge-ep-sandy-bridge-ep/) it's still not clear to me whether the new chips (specifically the 15-core dual-socket ones) will be compatible with the z9pe-d8/ws....
> 
> Anyone figure this out? 2x15 cores is a lot more than 2x8 for rendering....!
> 
> 
> 
> 
> 
> 
> 
> I'm aware that there are 12-cores that will work - e5-26xx v2 (provided I assume you install ecc-ram), but hey, losing out on an extra 6-cores. Of course hoping ES versions will be floating about (not really about to drop 12 grand on 2 processors)....
> 
> My 2x e5-2665 B0 are still doing fine, but any increase in rendering speed is welcomed...


They will cost a fortune and are keyed differently








Quote:


> Originally Posted by *shoek*
> 
> re: E7 processors... my understanding is that they use a version of Socket 2011 that is not pin-compatible with the E5 Xeon v1/v2 that the Z9PE-D8 uses.
> I'm not really sure what mobos will support the E7's -- probably something high end from Supermicro and the like
> 
> The Haswell EP Xeon processor should be out this fall using yet another (3rd) version of Socket 2011, also incompatible with the Z9PE-D8. I'm hoping Asus has a workstation dual processor motherboard for the Haswell Xeon that uses a more modern chipset than the Z9PE-D8's C602/X79 chipset.


Haswell EP and Ivy EX should be compatible (both LGA-2011-3)
Quote:


> Originally Posted by *YP5 Toronto*
> 
> This setup goes against all the posts of people have trouble with sli, non ecc memory, ect.
> 
> Damn..this build is sexy.


He is using an Ivy Bridge processor. Those don't have the same issues as Sandy B0


----------



## shoek

Quote:


> Originally Posted by *TheBlademaster01*
> 
> Haswell EP and Ivy EX should be compatible (both LGA-2011-3)


I was hoping Ivy EX and Haswell EP would use the same socket, but this article led me to believe otherwise:
http://www.servethehome.com/haswell-ep-ivy-bridge-ex-ivy-bridge-ep-sandy-bridge-ep/


----------



## TheBlademaster01

What socket is Ivy-EX then? Haswell is 2011-3 and we have 2011-0


----------



## 82ghost82

Quote:


> Originally Posted by *eyepiz*
> 
> Hi all!...I wanted to share my build, so far running rock solid.
> 
> 
> 
> 
> 
> 
> 
> 
> 
> It running high 60's under load, let me know if I should change out the CPU coolers.
> 
> Thanks!
> 
> Workstation Specs
> 
> Asus Z9PE-D8 WS (Bios 5304)
> (2) Intel Xeon 8 Core CPU's E5-2680v2 ES (QE5F)
> (32) Gigs of G.Skill Trident X 2400 running @ 1600
> (4) Evga Titan GTX in SLI
> (1) OCZ Vertex 4 SSD (OS)
> (1) Samsung Evo 840 (Data)
> Enermax Max Revo 1500W PSU
> Windows 8.1 Pro


Hi!
Nice workstation!
I would rotate the coolers by 90 degrees, so to not blow hot air directly on the ram.
I think they are R-17 models, so able to dissipate each till TDP 160W.


----------



## ZDngrfld

Quote:


> Originally Posted by *eyepiz*
> 
> Hi all!...I wanted to share my build, so far running rock solid.
> 
> 
> 
> 
> 
> 
> 
> 
> 
> It running high 60's under load, let me know if I should change out the CPU coolers.
> 
> Thanks!
> 
> Workstation Specs
> 
> Asus Z9PE-D8 WS (Bios 5304)
> (2) Intel Xeon 8 Core CPU's E5-2680v2 ES (QE5F)
> (32) Gigs of G.Skill Trident X 2400 running @ 1600
> (4) Evga Titan GTX in SLI
> (1) OCZ Vertex 4 SSD (OS)
> (1) Samsung Evo 840 (Data)
> Enermax Max Revo 1500W PSU
> Windows 8.1 Pro


Only 4 sticks of ram for both CPUs??? Boo dual channel.


----------



## YP5 Toronto

Quote:


> Originally Posted by *eyepiz*
> 
> Just ran valley.
> 
> It looks like GPU clocks don't boost to full speeds in SLI...weird anyone else running SLI NVidia cards?


Something is way off... you should be way higher...


----------



## squidman

Last build posted...eyepiz..OMG!!!

Awesome rig!







Shame Vray RT isn't optimized. Have you looked at Cuda-renderers such as Octane, Furry Ball (are they even around anymore?) or Arion (don't know how their cpu-gpu solution scales or whether they do animation). Ditto Indigo-render. Personally would love to mess about with Arnold, but have a feeling from what little info I can glean it's all CPU.

Octane must go like the clappers! sic...on four titans...if your workflow supports it (seems to be still somewhat beta)...still real-time rendering with 4xtitans....what a time saver. Even just for pre-vis...

Edit: too bad nvidia sucks for crypto-mining, otherwise they could pay for themselves....provided the whole market for altcoins doesn't implode...


----------



## squidman

Slightly different topic - back to storage on the z9pe-d8 ws...

Have whatever the hell it is around 30+ TB now stuffed into it, last 2 4tb hdd's on a separate pci controller. However I built a raid-5 using 4x4tb WD RED NAS drives using the last bottom SAS/SCU ports off the Marvell controller on the board. That gave me a bit over 10 TB? (writing from the laptop deep in the bush







of which I've already used 5, being backups of backups of...etc. Every video even from the mobile being HD not to mention pics just eats up space like no tomorrow.

So how do you migrate said raid array if you have to? What happens in the event of catastrophic system failure? Backing up 5tb isn't a major headache, (to two hdd), but when the array starts getting full (10 tb?)...any ideas?

Do people still use iodrives, or tape, or something (yes have talked with people about swapping out hdd's in safety-deposit boxes...impractical). Have unlimited storage online, but can't get more than 10 mbps up which makes that pretty useless (would really need 50 up).

Any ideas`? What happens if you have to disassemble pc? what happens to the raid array? Gone?

OK knocking on wood...at the moment if I had to sell/rebuild/reconfigure the beast, can't even do that without losing staggering amounts of data....









Wow it's great too see the old-skool smileys!







Missing the drinking buddies one tho'....


----------



## Jann

Quote:


> Originally Posted by *VitOne*
> 
> Thanks, if you have time, I'd like to see SolidWorks (usually runs on 1/2 cores while modelling, more for FEA, all for PhotoView 360) and other software like Modo.


Hey, I can't try Solidworks right now, but I did run Modo with some high poly teapots exported from 3dsmax (where I usually work).

3GHz.jpg 262k .jpg file

In 3dsmax for example I can't get anything higher than 2.9Ghz, so I suppose it uses 3-4 cores.
Hope this helps somehow


----------



## ZDngrfld

Quote:


> Originally Posted by *squidman*
> 
> Edit: too bad nvidia sucks for crypto-mining, otherwise they could pay for themselves....provided the whole market for altcoins doesn't implode...


Game's changed... You can get some nice hash rates now.


----------



## squidman

Not by nearly enough...have my 770 gtx, tried it a bit and got around 300kh/s, whereas a higher end radeon will get 8-900, and even a low end radeon will crush that (for altcoin/litecoin/scrypt mining). Using latest cudaminer off reddit. Even the new lowpower 750 ti reported on anandtech still gets only around 250 kh/s...fine iff you want to build a giant mining farm and electricity means something, but I pay .05cents/kw/hr, and don't want to invest in 12 or whatever new generation nvidia cards....just want to get my 3000 kh/s stuffed into my main machine...not bothering about electricity....so that basically means radeons (preferably a 7990)....

Probably too late already with dogecoins and noblecoins etc....it's all just a ponzi scheme anyway....gotta get in early...

that would be a problem anyway with this board (despite my giant 900d case)...can only fit 3-max 4 double-width boards (which is why people use the asrock board and pcie extenders in a milkcrate)...

Just don't have enough faith in this to invest 2-3 grand needed to make 3-4 kh/s....now if Cudaminer could start achieving 900 kh/s on titans (and wow 3-4 titans would make a hell of a cuda rendering machine)...then I could justify it.

In any case just spent 5k on fat rocker skis, the most insane bindings and boots and avvy bag....so rice and beans for a bit I think...at least they're REAL....wow freeride skiing is so friggin expensive....need a sponsor...anyone?









Edit: volkl one, dynafit beast, dynafit mercury, mammut pro protection 30l = 25000 nok or 4166 USD...yes we have 25% VAT or sales tax on top of fat margins.....(except for pc gear...YAY! there only 25% sales tax)...

Edit 2...sent from a Caravan park waaay in the mountains in Hallingdal, Norway where it's +7 centigrade and raining and slush....


----------



## ZDngrfld

That's just on scrypt based coins. Look into SHA-3 and the like.


----------



## squidman

don't you mean sha-256 and bitcoin-like alternatives? As that's dead as far as I can tell due to ASICS (and ASICS supposedly coming online for scrypt-based coins as well)...

gotta take a slash...what an expedition...on with down jacket, boots, crampons, headlamp...

this is all I found on sha-3 coins...https://cryptocointalk.com/forum/302-other-algo-cryptocoins/....can you refer to a specific coin and hashrates?

what pool(s) are you with?


----------



## TheBlademaster01

Scrypt-Jane


----------



## squidman

I assume you're referring to this?

__
https://www.reddit.com/r/1wskns/why_dogecoin_should_fork_to_scryptjane_and/

I still don't see a reference to sha-3 or any specific alt-coin, only a suggestion to fork as soon as possible before the evil asic-developers make inroads...still no way nvidia cards are even remotely viable...

Edit: mod...we should probably move this to a dedicated cryptocurrency thread..getting off-topic, my bad I started it!


----------



## squidman

So back to the raid conundrum-thing...

How to migrate? Any ideas? I never did get the intel or alternative solution to raid 0 the first two ssd's for OS and programs, finally put them on separate ssd's (samsungs 250gb)....what a pain. Because the raid controllers won't play nice with samsung....they just wouldn't install the OS in raid0 on boot (first two hdd sata controllers)...

Forget, am in the bush here, but it was either intel or marvell as far as I recall....and spent about a month on it....or you COULD raid0 the first two hdd's, but then you lost the rest of your sata ports or something as idiotic....

Edit: so how DO you stick say 4 titan black in there? Am on a measly 1200ax power supply, assume I will need a second......but will they even fit (see there could be problems with 4th card interfering with e.g. usb and sata/sas hubs).

Lastly (srry, ok also my last beer!), how would you connect a DAS with another say 8-12 hdd? Thought my old controller would do that, but no! Assume some kind of SAS controller, then breakout cable right out the back...or something....

cookie out, need to get ready for the pow pow...


----------



## eyepiz

Quote:


> Originally Posted by *82ghost82*
> 
> Hi!
> Nice workstation!
> I would rotate the coolers by 90 degrees, so to not blow hot air directly on the ram.
> I think they are R-17 models, so able to dissipate each till TDP 160W.


Good call...Thanks!

Quote:


> Originally Posted by *ZDngrfld*
> 
> Only 4 sticks of ram for both CPUs??? Boo dual channel.


Yea...I'm looking for a registered ECC 1866mhz that's 8X8 kits, so far can only find 16X16's I need 32 to 64 gigs max.

Quote:


> Originally Posted by *YP5 Toronto*
> 
> Something is way off... you should be way higher...


I think its driver related...the clocks don't boost all the way in SLI. My x79 board did with the same GPU's but not this board.


----------



## eyepiz

Quote:


> Originally Posted by *squidman*
> 
> Last build posted...eyepiz..OMG!!!
> 
> Awesome rig!
> 
> 
> 
> 
> 
> 
> 
> Shame Vray RT isn't optimized. Have you looked at Cuda-renderers such as Octane, Furry Ball (are they even around anymore?) or Arion (don't know how their cpu-gpu solution scales or whether they do animation). Ditto Indigo-render. Personally would love to mess about with Arnold, but have a feeling from what little info I can glean it's all CPU.
> 
> Octane must go like the clappers! sic...on four titans...if your workflow supports it (seems to be still somewhat beta)...still real-time rendering with 4xtitans....what a time saver. Even just for pre-vis...
> 
> Edit: too bad nvidia sucks for crypto-mining, otherwise they could pay for themselves....provided the whole market for altcoins doesn't implode...


Thanks!

Vray RT has gone a long way...I just upgraded to Vray 3.0 it seems to be getting there in terms of cuda rendering matching CPU production rendering. I've played with the others like Furry Ball and such, they seems to be VFX orientated, my workflow is Arch Visualization, Vray it seems to get better and better with every release. Corona looks promising but it's only CPU that I'm aware of. Redshift 3D looks promising and is coming to 3DS Max









What's crypto-mining btw?


----------



## YP5 Toronto

there should be some bios' by Skyn3t team. I use them on my 780s and they are safe when you follow the guidelines.


----------



## squidman

Crypto-mining is the illusion (or scam if you like) of creating alternative currencies in the hope that early adopters will become rich...with the proviso you are in it early enough. Late adopters can suck an egg. However it is pretty friggin' huge! That is for another time tho' (another story as my old man told me going to bed!). Look up the Dutch tulip mania....

Haven't tried Furry Ball, tried briefly Octane, but as it didn't work with vray or Vue or etc etc put me off...needs a completely new database of materials. However the speed isn't bad! Iff you can manage your workflow to fit that with multiple cuda cards, could be a real time-saver. Srry, mostly into vfx, so archviz etc is not my thing...but then for that would think the following should work really well...(Octane, Vray RT, Arion, Indigo etc). I guess it depends on whether you're doing animation or huge (8k) stills. Octane can do animation via a 3rd party plugin. 4xtitans should be more or less realtime rendering in octane would of thought. My beef with octane is yet another materials system, and no compatibility with e.g. Vue, Realflow etc...

Still a hobbyist here, so just my 2 cents worth...sorry iff that what incoherent, back from skiing after 3 hrs sleep! But what skiing! Snow is the next challenge for vfx.....(yes seen the demo!)...


----------



## 364958

Quote:


> Originally Posted by *squidman*
> 
> Crypto-mining is the illusion (or scam if you like) of creating alternative currencies in the hope that early adopters will become rich...with the proviso you are in it early enough. Late adopters can suck an egg. However it is pretty friggin' huge! That is for another time tho' (another story as my old man told me going to bed!). Look up the Dutch tulip mania....
> 
> Haven't tried Furry Ball, tried briefly Octane, but as it didn't work with vray or Vue or etc etc put me off...needs a completely new database of materials. However the speed isn't bad! Iff you can manage your workflow to fit that with multiple cuda cards, could be a real time-saver. Srry, mostly into vfx, so archviz etc is not my thing...but then for that would think the following should work really well...(Octane, Vray RT, Arion, Indigo etc). I guess it depends on whether you're doing animation or huge (8k) stills. Octane can do animation via a 3rd party plugin. 4xtitans should be more or less realtime rendering in octane would of thought. My beef with octane is yet another materials system, and no compatibility with e.g. Vue, Realflow etc...
> 
> Still a hobbyist here, so just my 2 cents worth...sorry iff that what incoherent, back from skiing after 3 hrs sleep! But what skiing! Snow is the next challenge for vfx.....(yes seen the demo!)...


If it wasn't for crypto currency, I wouldn't be able to afford my computer, my house (house hunting in NYC! ) nor anything else! All Hail The Crypto!


----------



## stenze

Hey all









I have a friend who bought this mobo, Asus Z9PE-D8 WS and soon getting some Xeon E-5 2620 V2 for it.
But the mobo is Rev. 1.04 (That means Bios 0503) and the CPU's is first on the list for Bios 5206.

Do he have a problem first boot?
Can he update bios with thoose CPU's, or without?
Or do he have to buy a third CPU for updating the bios?

Thanks guys


----------



## TheBlademaster01

It will not post on bios 0503 (this is not a guarantee for rev. 1.04 though, it could ship with any BIOS).

You can either buy a BIOS chip with 5206 or later (5304 is recommended). Or get a Sandy Bridge chip to flash the board.


----------



## MaDDeePee

Hello,

though this is my 1st post here i am reading since some month.

I got my way from Asus PC-DL (socket 604, fully modded) to socket 771 E5450 on a SM-X7DAL-E.
Now i am on the Z9PE-D8-WS train









I like to tell you that i was able to run:

1x Xeon E5-26xx-v2 (those super cheap chips from that ebay-guy called "upgradem*****") 8 core ES with 1,6Ghz / 2,0Ghz Turbo, 22nm ivy bridge ES - not present @ intel ARK with that specs (!)
on a new Z9PE-D8-WS with 5206 bios. A pair DIDN'T post until i flashed 5304. Since then everything is working fine, also in dual CPU mode.

I am running 8x Kingston KHX-2133-C11D3K4 / 8GX (total 16gb) and they are prime95 and memtest86+ stable at 1600Mhz.
I am unable to get bios posting higher than that, raising busclock >100 results in prime95 unstable...

Cool thread guys









Edit: Could anyone tell me where my mobo-rev is printed on the board?


----------



## TheBlademaster01

Here


----------



## lowfat

So I have about 45 days of non-stop folding w/ my Z9PE-D8 WS. Most stable system I have ever built. This is even w/ the tops of all the heatsinks plasti-dipped. They don't even get warm.









http://s18.photobucket.com/user/tulcakelume/media/Stacker/Z9PE-D8 WS/_MG_4059-1.jpg.html


----------



## TheBlademaster01

Which CPUs low?


----------



## lowfat

Quote:


> Originally Posted by *TheBlademaster01*
> 
> Which CPUs low?


2.3GHz 8-core engineering samples.


----------



## TheBlademaster01

Yeah, those aren't the hot ones. Looks nice (industrial). I saw your work some months ago and was impressed, but didn't know whether or not it would be a smart thing to do









You can always peel it off though, which is why I like the stuff. Too bad it's hard to get over here though


----------



## MunneY

Do you guys have any trouble running your ES CPUs... I'm thinking about joining the ranks


----------



## lowfat

Quote:


> Originally Posted by *MunneY*
> 
> Do you guys have any trouble running your ES CPUs... I'm thinking about joining the ranks


Lots of troubles. But once you get around them they can be good.


----------



## MunneY

Quote:


> Originally Posted by *lowfat*
> 
> Lots of troubles. But once you get around them they can be good.


Care to explain


----------



## TheBlademaster01

I recognize your username MunneY, but I don't know from where.

Anyways. Sandy Bridge B0 don't work properly with .CAP bioses (every BIOS release after 0703). Ivy Bridge only works from BIOS version 5004 and later, but 5004 only works with ECC RAM if you have an Ivy Chip. BIOS 5004 also completely breaks compatibility with Sandy B0, so they won't post anymore. QS Sandy Bridge C0 and later work with all BIOS revisions.

You also need BIOS 3506 and later to have the board recognize PCI-E 3.0 cards properly (Kepler, Tahiti, Maxwell etc.)

If you use PCI-E 2.0 cards only you could get away with B0 chips and absolutely not flashing to BIOS 3109 and later. After that it is hell (my situation







). It also won't work with Windows 8 on those older BIOSes.


----------



## MunneY

Quote:


> Originally Posted by *TheBlademaster01*
> 
> I recognize your username MunneY, but I don't know from where.
> 
> Anyways. Sandy Bridge B0 don't work properly with .CAP bioses (every BIOS release after 0703). Ivy Bridge only works from BIOS version 5004 and later, but 5004 only works with ECC RAM if you have an Ivy Chip. BIOS 5004 also completely breaks compatibility with Sandy B0, so they won't post anymore. QS Sandy Bridge C0 and later work with all BIOS revisions.
> 
> You also need BIOS 3506 and later to have the board recognize PCI-E 3.0 cards properly (Kepler, Tahiti, Maxwell etc.)
> 
> If you use PCI-E 2.0 cards only you could get away with B0 chips and absolutely not flashing to BIOS 3109 and later. After that it is hell (my situation
> 
> 
> 
> 
> 
> 
> 
> ). It also won't work with Windows 8 on those older BIOSes.


I was looking at these chips.

http://www.ebay.com/itm/2-Intel-Xeon-LGA2011-8-Cores-Dual-QPI-E5-2650-ES-QA82-QA8H-2-0GHz-20MB/261409064658?_trksid=p2045573.c100033.m2042&_trkparms=aid%3D111000%26algo%3DREC.RVI%26ao%3D1%26asc%3D20261%26meid%3D5373140329466218094%26pid%3D100033%26prg%3D9059%26rk%3D0%26rkt%3D4%26sd%3D261409064658

I will probably grab something like a 750Ti or something to use for video. Just gonna use it for a video on Youtube


----------



## TheBlademaster01

Not worth it. Those are B0 chips btw. Easy to recognize since they are QA82. QAxx are Sandy B0.

QBxx are C0/C1 (SRxxx = retail = C1/C2)
QDxx are Ivy B2 (not sure about the stepping, but they have no issues atm aside from lower clock bins/turbo than retail).
QExx/QFxx are C0/C1 Ivy IIRC.

But some QExx chips are far from retail as well. For example QE5F. Which is called E5-2680V2 in most ads but they are 8C/16T chips that turbo to 3.3GHz all core. But E5-2680V2 is a 10C/20T chip that turbos to 3.1GHz all core.

Same for some 2670V2 and 2690V2 chips.

In your price range I'd probably buy these

http://www.ebay.com/itm/Ivy-Bridge-EP-QDNR-ES-LGA2011-2-3Ghz-2-7Ghz-Turbo-10C20HT-25M-L3-L1-Tested-/231176733239?pt=CPUs&hash=item35d3350237

2.7GHz all core 10C/20T and Ivy Bridge B2.


----------



## lowfat

Quote:


> Originally Posted by *TheBlademaster01*
> 
> http://www.ebay.com/itm/Ivy-Bridge-EP-QDNR-ES-LGA2011-2-3Ghz-2-7Ghz-Turbo-10C20HT-25M-L3-L1-Tested-/231176733239?pt=CPUs&hash=item35d3350237
> 
> 2.7GHz all core 10C/20T and Ivy Bridge B2.


His link was for 2 CPUs


----------



## ZDngrfld

My QDNRs are running great in my Z9. 106 BCLK, 2860MHz all-core turbo. I've been thinking of picking up one of those QE5Fs or something similar for a new 1P gaming setup. Tired of this SR-2.


----------



## TheBlademaster01

Quote:


> Originally Posted by *lowfat*
> 
> His link was for 2 CPUs












My bad.

Still, he's probably going to get issues in combination with that board because they are B0.


----------



## MunneY

Quote:


> Originally Posted by *TheBlademaster01*
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> My bad.
> 
> Still, he's probably going to get issues in combination with that board because they are B0.


I'm not hard set on the board, I just wanna get something thats gonna work!


----------



## ZDngrfld

I ran the QA82s on my Z9. Albeit with the 0503 BIOS and no video card. You'll have issues with newer GPUs like the GTX 750 Ti. I'd take TheBlademaster01's advice and look for some newer chips.


----------



## TheBlademaster01

Quote:


> Originally Posted by *MunneY*
> 
> I'm not hard set on the board, I just wanna get something thats gonna work!


Probably best off with a Supermicro board then if you plan to get those chips. Otherwise QBxx chips or QDxx/QExx/QFxx Ivy chips.

Quote:


> Originally Posted by *ZDngrfld*
> 
> I ran the QA82s on my Z9. Albeit with the 0503 BIOS and no video card. You'll have issues with newer GPUs like the GTX 750 Ti. I'd take TheBlademaster01's advice and look for some newer chips.


How are those B0 chips in your X9DRL?


----------



## ZDngrfld

Quote:


> Originally Posted by *TheBlademaster01*
> 
> How are those B0 chips in your X9DRL?


They're working fine. No clue what version BIOS I'm running on it, though.


----------



## squidman

Haven't been back for a while (been skiing powder in Norway!) so back to my thread regarding the mobo (z9pe-d8 ws) and the max cpu's we can plop in there...I take it that it is only as far as xeon v2's and that they require ecc memory? As I'm still on b0 stepping e5-2665's, which are ok but as soon as I start doing some realflow **** or even simple gopro studio rendering, wellll....still not fast enough! At least the v2's have a 12? core out there, but not too cheap on ebay yet...

Any ideas on how far we can go with this board? And what would it take (memory etc and spicy chips on flea-bay)?...

Wow how is it rendering is STILL too slow (coming from an i7 laptop mind you....prbly just forgot how rendering/simulations used to take a week or thereabouts....)...?









Edit: What is the fastest (given future developments) we can take this board before point of diminishing returns? Haven't seen an lga-2011 board yet from supermicro or asus that supports e7 chips....apparently not compatible with lga-2011 v1 or 2 or whatever it is that supports our socket....


----------



## MaDDeePee

I got 22nm ES chips that are working without ECC ram.


----------



## squidman

Drool...! Some specs or stepping details? NEED something faster....my 2665's prbly run around 2.2 ghz max....nice with all the threads but not really cutting it in realflow or vue...ditto simple h.264 rendering for gopro vids....dunno iff going to say an e5-2695 v2 is worth it..(?% faster in Cinebench`? Get around 19 on my setup now...)..but for 2 still around 2k usd even ES on fleabay...(800 usd each +shipping * sales tax 25%)...would go from 19 in cinebench to maybe 27.......at a cost of 2 grand...


----------



## TheBlademaster01

Quote:


> Originally Posted by *ZDngrfld*
> 
> They're working fine. No clue what version BIOS I'm running on it, though.


Even with PCI-E 3.0 GPUs?

Quote:


> Originally Posted by *squidman*
> 
> Haven't been back for a while (been skiing powder in Norway!) so back to my thread regarding the mobo (z9pe-d8 ws) and the max cpu's we can plop in there...I take it that it is only as far as xeon v2's and that they require ecc memory? As I'm still on b0 stepping e5-2665's, which are ok but as soon as I start doing some realflow **** or even simple gopro studio rendering, wellll....still not fast enough! At least the v2's have a 12? core out there, but not too cheap on ebay yet...
> 
> Any ideas on how far we can go with this board? And what would it take (memory etc and spicy chips on flea-bay)?...
> 
> Wow how is it rendering is STILL too slow (coming from an i7 laptop mind you....prbly just forgot how rendering/simulations used to take a week or thereabouts....)...?
> 
> 
> 
> 
> 
> 
> 
> 
> 
> Edit: What is the fastest (given future developments) we can take this board before point of diminishing returns? Haven't seen an lga-2011 board yet from supermicro or asus that supports e7 chips....apparently not compatible with lga-2011 v1 or 2 or whatever it is that supports our socket....


Maximum will be E5-2697v2

Xeon E7 v2 and E5 v3 will not be compatible.

Quote:


> Originally Posted by *MaDDeePee*
> 
> I got 22nm ES chips that are working without ECC ram.


Yes, only 5004 requires ECC in combination with Ivy chips.
Quote:


> Originally Posted by *squidman*
> 
> Drool...! Some specs or stepping details? NEED something faster....my 2665's prbly run around 2.2 ghz max....nice with all the threads but not really cutting it in realflow or vue...ditto simple h.264 rendering for gopro vids....dunno iff going to say an e5-2695 v2 is worth it..(?% faster in Cinebench`? Get around 19 on my setup now...)..but for 2 still around 2k usd even ES on fleabay...(800 usd each +shipping * sales tax 25%)...would go from 19 in cinebench to maybe 27.......at a cost of 2 grand...


E5-2697v2 gets 17 in cinebench for 1 CPU from what I've seen. E5-2695v2 gets 16. E5-2690v2 gets 15 ish and E5-2680v2 gets around 14.

In 2P it's around 1.95x those numbers. Not sure where those 5% losses come from though.

I was considering making a list of performance and ES chips. But it seemed to be such a tedious job and it didn't seem people were interested in it


----------



## ZDngrfld

Quote:


> Originally Posted by *TheBlademaster01*
> 
> Even with PCI-E 3.0 GPUs?


No clue. I don't have video cards (or a GUI for that matter) in any of my rigs except for my SR-2


----------



## TheBlademaster01

Okay. Yeah, my B0 chips are fine as well in my Z9 with PCI-E 2.0 cards


----------



## phillydee

Hi all. I'm trying to solve my latency issues caused by very high DPC. I have dual Xeon E5-2665 ES QB7U (is this C0 stepping? I can't remember) running on BIOS 5004.

Using LatencyMon, I am seeing that it's storahci.sys that's causing the issue. However, every time I use INTEL's AHCI driver and/or install Intel RSTe to use Intel's AHCI drivers, Windows fails to boot up and it just gets stuck on the windows logo.

Soooh..... I'm now thinking I should update the BIOS and try installing IRSTe again. My only concern is if the newest BIOS (53xx?) would play nice with my ES chips.

Any help and/or advice would be extremely helpful... thanks!


----------



## TheBlademaster01

Newest BIOS should be fine with those chips.

I can't help you with your storage problems however


----------



## phillydee

Thanks... I am using a Samsung 840 Pro SSD (512) and really shouldn't be having any issues and for the most part evrything is fine except for the damned system interrupts that causes audio to stutter, choke, and have dropouts.

I'll give the newest BIOS a go. Thanks.

BTW does anyone else use the Intel Rapid Storage Technology enterprise?


----------



## TheBlademaster01

5004 was kind of a dud. It needed ECC RAM to get Ivy to post and doesn't post with sandy B0 at all.


----------



## daqe87

Hello guys. I'm new here and a fresh owner of this mb. I try to find similar problem that bothers me in this thread, but after 10 pages reading I gave up







.

My configuration:

-Asus Z9PE-D8 WS
- 2 x Intel Xeon E5- 2670v2
- 8x4gb Kingston Kingston HyperX KHX2400C11D3/4GX (1600Mhz 9-9-9-24 1T)
- 850W Cooler Master M2Pro
- Samsung EVO 840 120GB
- OS is Win7 Ultimate 64bit

My problem is cpu's turbo boost. Boost increase only up to 2.9GHz and E5-2670v2 shoul have boost up to 3.3GHz. Here is some pics:




Bios version of mobo is 5304. In bios stands "Target cpu turbo mode speed: 3300MHz" , but when system starts up, cpus always reach only 2.9GHz (during bench testing, rendering...)
You can see in cpu-z that multiplier range is 12-33, but both cpus reach maximum 29. I started system with one cpu only , but nothing changes.
What is the problem? Is there any bios settings I'll need to do or what? Please help


----------



## TheBlademaster01

3.3GHz is single/dual thread turbo. 2.9GHz is the maximum all core turbo.

If you would run the single threaded bench for example you should see 3.3GHz

That score looks fine to me though. 25p is about what I would expect from those chips (comparable to 2687Ws)


----------



## phillydee

Quote:


> Originally Posted by *TheBlademaster01*
> 
> 5004 was kind of a dud. It needed ECC RAM to get Ivy to post and doesn't post with sandy B0 at all.


Just to be sure: the 5304.cap bios should be fine in my machine currently on the 5004.... correct? I can't recall what format the bios was when I last updated. I haven't flashed yet as I will probably re-set up windows so I just wanted to double check.... thanks.


----------



## TheBlademaster01

All releases past 3109 are .CAP. And yes, they should work with Sandy C0/C1 (QBxx)


----------



## phillydee

OK thanks--can't be too careful...


----------



## phillydee

OK, after flashing the BIOS, installing the Intel Rapid Storage Technology (IRSTe) software stil caused Windows to hang on boot. I then changed the jumper from the default LSI MegaRAID to the Intel RSTe. Even though I have no RAID set up, it seemed this was key to both installing the IRSTe software/Intel ACHI drivers and eliminating the system interrupts which caused me to not be able to work with any audio without serious stutter and lag.

I appreciate the help, I'm posting the above in case anyone else searches for solutions in a similar scenario.


----------



## squidman

I was never able to get irste or lsi raid0 to work on the first two sata 6 ports (for two samsung ssd's)...turns out lsi doesn't support my older samsung ssd's amongst other things. I WAS however to get the LSI equivalent (ssd caching) to work on 4 other ports, and installed a raid card which shows up under the same lsi control panel (believe it or not ran out of sata ports! well I have a lot of hdd's, something like 16). My main problem was getting win7 to install with the first to sata ports set to raid0....just wasn't going to happen.

It's sort of stable at the moment, but even booting from an ssd for the os, takes 10 mins to start (maybe I should have avoided installing the irst-e(enterprise) thing...at this point, it just stays on, don't really want to mess with it. The last 4 sata/scu ports work fine as well in raid5 (4 WD red drives) which is nice...don't know what happens if the system crashes of course...









Edit: you're not able to have raid0 on the first topmost sata ports AND JBOD for the rest...so I guess the solution if you want to raid0 som ssd's for the OS and programs (much easier backup-wise) is to install a raid card and run OS and programs off that, then use the board's ports for variously lsi-mega raid, the intel scu/raid, jbod etc...

At this point have os on one 250gb samsung ssd, programs on another (the top 2 sata6 ports), then the next 4 using lsi ssd caching (2 cheap 64gb ssd's paired to 4tb hdd's), then a whole lot of jbod disks, then raid 5 on the intel scu ports, then another 2 on the raid controller. Would like to use the raid cart for port-multiplier, but easier said than done...of course can only fit another 2-3 hdd's in my 900d case (have big ax1200 power supply which gets in the way of the 5th hdd cage)...

whew can't have enough backup/storage!







Of course i'm evacuating bricks when it comes to backup...have only 10 Mbps up...


----------



## phillydee

Yeah after a day of thinking everything was smooth running, my system got slower and slower until it just wouldn't do anything. I'm kind of back to square one--do I live with my constant system interrupts/high DPCs or do I continue to troubleshoot... at this point I'm content to turn this system into a render machine and just build an X79 system for workstation use.


----------



## MaDDeePee

BIOS 5404 for Z9PE-D8 WS released to public
1. Improve the FAN curve settings.
2. Patch the compatibility for K4000.

[Warning]
*If your BIOS version is 0703 or older: Go to BIOS-Utilities to download and flash the BIOS to version 3109 firstly before updating the BIOS.

4,15 MBytes 2014.03.14 update

Anyone?









I guess i can't expect any other benefits from this bios (like RAM speed compatibility), just the two fixes told there?


----------



## squidman

Phillydee (mmm puts in the mood for a philly cheesesteak! but how to find in norway?)...

Yeah the slowing down thing is an absolute nightmare on bootup. But with various raid systems in place and terabytes upon terabytes of data, am EXTREMELY reluctant to reinstall everything...what a nightmare eh! Not to mention all the plugins....

It never did boot really quickly even on fresh install being a workstation, but getting pretty bad for me lately...just have to take an extended extended coffee break.

Of course for rendering, video stabilizing or those kinds of things, it just ROCKS!


----------



## 2disbetter

Quote:


> Originally Posted by *t_ski*
> 
> Well, I tried both CPUs and it didn't work (I really didn't expect it to be any different). Then I took the cables from my AX1200 and hooked them up to the board. I pressed the power button and too my surprise I saw both CPU fans spinning!!! Then I heard crackling, saw smoke, then a small fireball appeared under the heatpipe on the board.
> 
> 
> 
> 
> 
> 
> 
> 
> 
> Going to have to RMA for sure. Not sure if I can RMA through Newegg now or if it will have to go through Asus.


I was going to tell you that your problem was most likely your PSU. Seriously under powered for what you were running. Same thing happened to me. Upgraded the PSU (1300 watts) and everything fired right up. Your little fireball could have been a result of using the wrong cables and shorting your board. Hopefully you got an RMA and new PSU.

2d


----------



## squidman

What on earth was he running? I have the same older ax1200 PS, running (ok just the one 770 gtx), but loads of fans, 13 hdd or something, raid card, 2 h80i's, and looking at my watt-meter only pulling 230 watts on idle. It went up to something like 400 watts when maxing out both cpu's and gpu alt-coin mining...(back in the day sic! all of one month ago before market imploded). Seen several vids with this board and e5-2697 v2's and the same PS running fine (albeit mosfet heatsinks get a bit warm....link here...



)...

hahah sorry just love the guy's accent and occasional off-color language! Legend! Mind you Heathrow is still a friggin' nightmare...Gatwick all the way for me....


----------



## 2disbetter

He had dual xeons, a either 280 or 770ti in it and a couple of hard drives, and 32gb of ram. He was trying to run it all with a 650 watt PSU, that he swapped cables with from his corsair 1200.


----------



## squidman

Why not chuck in the ax1200 to start with? Oh well, I don't know the whole story...I guess the lesson is go on the forums and do your research beforehand! Wow was grateful for this forum during my build! Still, even a 650 watt PS should have been able to handle the afore-mentioned. Sounds like a short, or a bad cable or maybe a stand-off. Really bad luck iff it all went south....commiserations!

Edit the 'hahaha' of last post is from the vid I posted, nothing to do with this dude's problems...that just sucks! And is not funny at all, really sympathize with the guy, hope he can sort his probs...have had my share of pc-nightmares (and wow with this board, ya know, prbly more to come knock on wood).

Good luck t-ski! May you have endless powder dump on you as a recompense for your tribulations! From another skier...


----------



## t_ski

The story is that the AX1200 is in my main rig, currently running with a 3930K/X79 combo I wanted to get the Asus up and running first, then transfer over my files and replace the system in my main rig. I was using an Antec 650W which should have had enough amps on the rails to work, but I think it was not EPS 2.9 compliant, which this board seemed to need. Anyway, I took the AX1200 and tried to power up this board with it, but I used a spare ATX cable I had bought second-hand. I now think the other guy tried to sleeve it and messed up the pin-outs, which fried my board.

Update on the story is that Newegg got my board and said the socket was damaged, so they returned it to me. No $hit - it fried my CPUs! I've had it sitting in the box since it came back from the Egg, unopened. I did get a chance to stop by a Microcenter and picked up a Sabertooth X79 board. I tried each of the Xeon chips in it and both give a red CPU error LED. Long story short: over $1500 down the toilet...








Quote:


> Originally Posted by *squidman*
> 
> Good luck t-ski! May you have endless powder dump on you as a recompense for your tribulations! From another skier...


I don't ski - I'm Polish


----------



## squidman

Dude, you now really qualify for some good karma! Or something! Well 1500 bucks isn't too too bad, my apartment complex is down around 100k after neighbor did some dodgy roof-work, and my family is sitting with about 60% of that (so around 60k us) with no resolution in sight and insurance companies running away and lawsuits and and...

Or you could be a Ukrainian soldier in Crimea...etc...so all in perspective as they say....!

But still think you qualify for some NATO reassurance! ?? There's lot's of great skiing areas in Poland, ever heard of Justyna Kowalczyk? I thought she was your national hero?

Well lets' all hope Putin doesn't get too greedy or bent on empire, otherwise all this will be pretty friggin' academic...









Remember always (whether Chinese or European..! dumplings in the freezer whether gyoza or piroshki or samosas...gotta have'em)....damn I'm out...


----------



## Science

Hi guys,

I am in the middle (rather at the end) of collecting stuff for 2 socket system
I read this thread from page 100 to the end, and from the beginning to 10. It is too much for me









The system:

1 x CaseLabs STH10
2 x Super Flower 1000W Platinum
1 x Asus Z9PE-D8 WS
2 x XEON E5-2687W v2 or E5-2667v2
2 x 64GB Kit* (4x16GB) - DDR3 1600MHz KVR16R11D4K4/64 (128 GB ECC Reg. Kingston 1.5V)
4 x GTX 780TI
watercooled (besides motherboard and RAM)
noiseless (inaudible) even at full load
Purpose ? Scientific analyses with multi-threading and CUDA, OpenCL.
I don't need 4way SLI, but I will try from time to time to play with Nvidia Surround. But I don't have a time to play games. Only Starcraft

My questions:

Which CPU should i choose??
2687W v2 has 150 TDP, but this is a box version (3 Y warranty)
2667 v2 is 130 TDP but this is a tray version







(1 y warranty
I know the differences between them, but I would like to hear your opinion.
I will try to rise bclk a little. How far I can go? 104 or 110?
Do you have any experiance with watercooling this mobo using EK waterblock? From their page and from the Internet I know that there may be some troubles when using 4 GPUs.... But this chip and VRMs should be cool, right?
It would be nice to hear your opinion.

Kind regards


----------



## TheBlademaster01

Depends on the pricing. There's no reason to pick a 2687 over a 2667 or vice versa so go with whatever has the best price. CPUs don't die so easily either but for the same price pick the one with the best warranty.

Count on 104MHz bClk for those obscene amount of GPUs (nice system btw, drool-worthy







)

Haven't tried watercooling this board because I mainly don't see the point in doing that but I understand why you would do that







. I also watercool the CPUs while they don't really need it. I would advise you to first test the system on air since this board has quite the DOA rate. Then when you verified functionality you can drown everything to your heart's content.


----------



## Science

Thanks for your advices
Quote:


> Depends on the pricing


the difference in price is about 50 EUR - so it is nothing.
Quote:


> CPUs don't die so easily either but for the same price pick the one with the best warranty.


hmmm So for me it is a choice between 150 TDP and 130 TDP








Quote:


> Count on 104MHz bClk


grrr.














Why we don't get what we would like to get







???








Quote:


> nice system btw


at this moment almost all stuff is on the paper







I will buy all components in about 3-4 weeks. It is not so simple








Quote:


> I mainly don't see the point in doing that


me too. I saw a review (unfortunately I don't have a link) with pictures where this chip and VRM is cool at full load. But still... It would be awesome to use EK waterblocks







Just for fun.
Quote:


> while they don't really need it


Yes, but how much space you save using WC ?







IMO watercooling is a future.
Quote:


> I would advise you to first test the system on air since this board has quite the DOA rate


Yes, I always do this. And from this thread I know the DOA is a problem.

Once again thanks, maybe someone els would like to say something?


----------



## TheBlademaster01

I also don't really know what deal you can get on those chips, but unless you need that 3.6GHz clock you should get 2680V2 since they are somewhat faster overall.

E:

Yes, I did it for looks and actually the sheer ease of it (replacing CPUs etc.)


----------



## Science

http://ark.intel.com/products/75277/Intel-Xeon-Processor-E5-2680-v2-25M-Cache-2_80-GHz

It is slower than

http://ark.intel.com/products/75273/Intel-Xeon-Processor-E5-2667-v2-25M-Cache-3_30-GHz

Maybe you lost one "2"







? and you thought you wrote

http://ark.intel.com/products/76161/Intel-Xeon-Processor-E5-2687W-v2-25M-Cache-3_40-GHz








?

2687 and 2667 are the same overall when it comes to power - as far as I read in The Internet. It is because they have exact the same speed with addition of turbo.

E5-2667 v2 base clock: 3.3 turbo mode: 3/3/3/3/4/5/6/7
E5-2687W v2 base clock : 3.4 turbo: 2/2/2/2/3/4/5/6

So they are the same.


----------



## TheBlademaster01

No I mean that 2680V2 you listed









E5-2680V2 turbos to 3.1GHz all core (10 cores) vs E5-2687WV2/E5-2667V2 that turbos to 3.6GHz all core (8 cores). Or E5-2690V2 that turbos to 3.3GHz all core (10 cores).

In performance it is E5-2697V2 > E5-2695V2 > E5-2690V2 > E5-2680V2 > E5-2687WV2 = E5-2667V2 ~ E5-2670V2 > E5-2687W > E5-2690 > E5-2660V2 etc.


----------



## 2disbetter

Here is a serious question that will most likely expose my ignorance of the Xeon world, but I am currently running two E5-2609 v2s. Based on what I've seen and read my idea to upgrade to a set of E5-2630 v2 chips makes since. They do cost twice as much so the improvement is a no brain-er.

My question is, for this amount of improvement/cost is it worth it upgrade to this chip?

The pros as I see em are:
-6 cores vs 4
-HT (12 vs 4(ie: none))
-100 Mhz faster
-Able to turbo to 3.1Ghz
-Support 1600 Ram (which I'm using) instead of only 1333

cons:
price (but still significantly cheaper than most of the chips you guys seem to be running.)

2d


----------



## Science

Quote:


> Originally Posted by *TheBlademaster01*
> 
> No I mean that 2680V2 you listed
> 
> 
> 
> 
> 
> 
> 
> 
> 
> E5-2680V2 turbos to 3.1GHz all core (10 cores) vs E5-2687WV2/E5-2667V2 that turbos to 3.6GHz all core (8 cores). Or E5-2690V2 that turbos to 3.3GHz all core (10 cores).
> 
> In performance it is E5-2697V2 > E5-2695V2 > E5-2690V2 > E5-2680V2 > E5-2687WV2 = E5-2667V2 ~ E5-2670V2 > E5-2687W > E5-2690 > E5-2660V2 etc.


OK OK now I understand what you mean... Overall performance... You know.. I am still in the world of One core performance

But I will perform some benchmarks to measure if 10/12 core Xeon with lower one core performance will be a better choice.

I have to switch to double socket Xeons - I have no choice. But, still I need some high one core power (but I will check it). In a perfect world I would like to have the most cores and the higher per core power







We know it is impossible.

IMO if more people use real watercooling (not closed loop) Intel would like to make higher TDP chips. ... For example 250 or more TDP: 8 cores, with unlocked multiplier, so we can OC it to 4.4 or higher.
Closed loop WC is good, because people may turn into real watercooling









Unfortunately I need both more cores and highest possible performance per one core. That is why I am interested in 8 core Xeons, because IMO they are perfect for me.

But, because you have turned my attention to other possibilities.... I have to consider 10-12 core Xeons

Kind regards


----------



## Science

I made a simple comparison of interesting models:

Code:



Code:


Model        Base_clck       1_core_[GHz]    All_cores_[GHz] TDP     Cores   Power_*1        Power/TDP
2680v2  2.8     3.6     3.1     115     10      31      0.27
2690v2  3       3.6     3.3     130     10      33      0.25
2697v2  2.7     3.5     3       130     12      36      0.28
2667v2  3.3     4       3.6     130     8       28.8    0.22
2687Wv2 3.4     4       3.6     150     8       28.8    0.19
2643v2  3.5     3.8     3.6     130     6       21.6    0.17
3770k*2 3.5     4.6     4.6     ~150    4       18.4    0.12

*1 - All cores frequency x core count                                                   
*2 - an overclocked example

The same but as an image:



So I am confused right now... More One core Power or more overall power ??







I have to make some tests.

The darker the colour, the better


----------



## TheBlademaster01

Same thing I'm thinking about. I'm wondering if I should buy 2680V2 or 2695V2. I'd like to have some power per thread, but obviously overall power is important to me. For now I'm set on the 2695V2

Maybe this chart will help as well.



@2d

I would say it is worth it. What work are you doing on the rig?


----------



## Science

Quote:


> Originally Posted by *TheBlademaster01*
> 
> Same thing I'm thinking about. I'm wondering if I should buy 2680V2 or 2695V2.(..)


TheBlademaster01 I MUST thank you. You opened my mind! Seriously, I thought I had everything set, but it turned out, there are better options than 8 core Xeon...

I made quick benchmarks... I set my 3770k to 3 GHz. No turbo, only 3GHz for all cores... I don't need high GHz per core! This is my first conclusion.... I am shocked... Seriously....
We are talking about 3 GHz vs. 4.6 GHz..... The system (GNU/Linux or even Windows 7) is responsive both when CPU is set to 4.6 or 3 GHz ! One core applications are slower, but we are talking about differences in... 0.5 - 2 seconds ??







And it is on old, but still one of the best Samsung 830







.

Once again. TheBlademaster01....... gosh... THANKS Now i consider 10 or 12 core Xeons. And I don't care much about high one core turbo any more







I am glad I found this thread and I sign for this forum!









I will perform more tests.

Best Regards!


----------



## TheBlademaster01

Glad I could help









It's mainly Intel's numbering scheme that doesn't quite make sense. 2687WV2 is a fast processor but with the V2 chips there are quite a lot of faster chips (and cheaper + more efficient ones). I think efficiency and stability are some of the most important features of these platforms.


----------



## Science

Yes, I agree: it is a naming mess.

Can you please explain me, should I care about double memory controller inside 12c Xeons? Is there any drawback of it? I assume, I should populate all RAM slots in this mobo when using 12c Xeons... Am I right?

I am thinking about E5-2697 v2... The only thing which is scary is the price.... But I have one day to make a choice!








One thing is certain: No more over-clocking. It is useless in this case.

I wonder if Intel has its own way to find out if the processor was overclocked.


----------



## TheBlademaster01

Double memory controller?

You should populate all DIMMs, but each CPU has a quad channel memory controller. You can however decide whether you want the CPUs to be able to access RAM independently over their own memory bus, or have them fight for bandwidth over the same bus. This can be done in the BIOS by enabling NUMA or disabling it respectively. You should always experiment with which works best for your application. Some applications prefer node aware memory access while others work better when addressed uniformly (or rather randomly) <- typically bad multiprocessor programming. I generally leave NUMA on, it gives you a lot of options within the OS to configure things (not quite as good on Windows 7/8 vs something like Windows Server or Linux distros like Red Hat etc.)

You can't really see if the CPU was overclocked. Just keep in mind that if you overvolt the chips and burn them, they'll likely find out by inspecting the condition of the chip. I think that they should still honor an RMA if you keep that information a secret though









But overvolting the chips by that much is completely pointless for these processors since you are limited to bClk overclocking. Which typically ends up being a PCI-E bus/controller limitation


----------



## Science

What can I say







Your knowledge is priceless







Thank you, because my Internet investigation was unsuccessful.
You saved me a lot of time


----------



## TheBlademaster01

Once again, glad I could help







. I learned most things in this thread or experimentation as well. It's an interesting platform to work on. You just have to keep in mind that if you take the common road (meaning single processor systems) you are guaranteed to encounter problems, since the majority of the applications can not handle that much cores/nodes (nodes being NUMA nodes or freely translated as 'sockets').

E:

Before I forget to mention it. Make sure your motherboard comes shipped with BIOS version 5206 or later or Ivy processors will not post)


----------



## Science

Quote:


> experimentation as well










I do like to experiment too, but... When I buy this machine, I wont do ANYTHING crazy! You know.., I invest A LOT of money







I don't want to end up crying








Quote:


> You just have to keep in mind that if you take the common road (meaning single processor systems) you are guaranteed to encounter problems, since the majority of the applications can not handle that much cores/nodes (nodes being NUMA nodes or freely translated as 'sockets').


Can you please explain? I don't understand... I will use 2 of these 12c Xeons if I make a final decision.
Linux is well threaded. Software I use is well threaded too.
Is there something more hiding behind NUMA than just memory management?

Code:



Code:


Make sure your motherboard comes shipped with BIOS version 5206 or later or Ivy processors will not post

It may be difficult, but I will try. Tomorrow I will call here and there.

The original BIOS version is related to Motherboard version? The version is printed on the silicon? Am I right?

Hesus







I have bought tons of motherboards in my life and I NEVER was so afraid that something will go wrong







This mobo is very mysterious.

IMO if Asus put v2 (Ivy) CPUs on their QVL it is no way I will end up with no boot







But I know from this thread a lot of people had this issue.


----------



## TheBlademaster01

It depends on how long the board has been on the shelf. You can order pre flashed BIOS chips from ebay if it doesn't post though. But you could ask the seller/store to flash the BIOS for you. It will not post on versions before 5xxx series BIOSes and Ivy chips. 5004 will post with Ivy chips but only in combination with ECC ram. 5206 should post on any ivy configuration.

Sadly revision number is not enough to make a conclusion









I was more meaning that some games in Windows will not run without issues on these systems. Though typically these problems are solvable. I haven't encountered issues on Linux. You can install numactl to gain control over your nodes and thread bind applications. Works perfectly


----------



## 2disbetter

Quote:


> Originally Posted by *TheBlademaster01*
> 
> ...
> 
> @2d
> 
> I would say it is worth it. What work are you doing on the rig?


Mainly Windows and Android game development. (So running 1 to 2 vms, 3d art work, sound design, etc) Nothing that really requires this type system, but what can i say? I'm an enthusiast. So id like to have good parralel power incase my vm requirements increase and for the rendering advantages, but also want a component gaming machine as well. The 2 2609s are already see to that. So this upgrade is really just to future proof some and improve performance. (based on my cinebench score of 698 vs a 2630 1538 score i can see that it would be quiet the improvement.)

2d


----------



## TheBlademaster01

I think those chips would be very much worth it if you can afford them







.

You could definitely use the extra cores for the VMs. I really like this platform for how multifunctional it is and only pulling 280W from the wall peak (my Sandy chips). I'd imagine that your Ivy chips would do much better. Go for it


----------



## happy hopping

I bought this brand new in Dec. 2013. There are 2 issues, 1 hasn't surface since Jan. So I want to talk about the current issue:

about 3 weeks ago, after I push in a magic jack, and I have to move my 2nd USB keyboard plug because of it, it makes my USB keyboard light faded out of the system. In other words, I can't access my keyboard.

This keyboard needs 2 USB port to power:

http://www.tesorotec.com/gaming-keyboard/tesoro-colada-evil-g3nl-aluminum-backlit-mechanical-gaming-keyboard.html

so the keyboard manufacturer ask me to update the Keyboard BIOS, and the driver and everything works fine since.

Last night, I push in my USB 3.0 CF card reader to the front port of my case, the case is a NZXT Phantom 820 and there is no response. so I push that same Cf card reader to my USB 2.0 front port, and the PC is able to see the CF card. But that USB 3.0 port was working since Dec. 2013.

Then right after that, the light on the USB keyboard faded out for a moment. i.e., the keyboard lost connection w/ the system again.

ANd just now, the same thing happens, almost as if there is a shortage of electricity to the keyboard.

The keyboard is connected directly to the rear of the Z9PE-D8 on the first 2 USB ports.

The USB driver is the 1 that c/w the Asus CD, it says ASMeida XHCI controller, plus the standard USB driver that c/w microsoft

Should I flash the CMOS or update the asus USB driver? Or do you think this is some sort of static electricity issue due to this time of the year?

========================================

The other issue that has somehow go away since Jan. 15 is:

On somewhat rare occasions, there is blinking cursor at the top left corner of the screen right after the Asus welcome screen, and it just hangs there. In the past weeks (Dec. 2013), I only seen that problem during a re-boot. But on Jan. 15, I see it during a cold boot. But it hasn't happen since, so I didn't do anything to it.

====================================

My set up is:

Z9PE-D8 w/ the Oct. 2013 BIOS flash
4 x 4 GB of Kingston non-register Hyper X RAM
Win 7 Pro, SP1
Quad. core Xeon 2.8GHz x 2
1 x Asus GTX 650 Ti Boost on Slot 2
1 x Creative ZxR

Nocuta Heatsink / fan on both CPU with electricity connects to CPU fan No. 1 & 2

The remaining sys. fan is connected to the fan controller of my NZXT Phantom 820 case, so none of the sys. fan port were used

The USB port in the rear of the motherboard are connected to:

2 x USB port for this Tesoro Keyboard
1 x USB port connect to a MMO 7 Mouse
1 x USB 3.0 port connects to a Kingston flash drive

The magic jack is never connect, and haven't connect since the above incident

Everything is brand new purchased in Dec. 2013


----------



## TheBlademaster01

I must say that these issues are new to me. What model Xeons are they? (if ES what Qxxx number?). Also need to know the exact BIOS version (can be found in main BIOS screen).

What does the motherboard led display if it hangs?


----------



## happy hopping

the xeon is E5-2609, I'll dig out the BIOS Ver. later, for now, it should be the latest, the Oct. 2013 ver., I'll let you know. As to that cursor hang problem, that is back in Jan., I didn't check the Q code, I should have, see at the time, I thought it's a nvidia driver acting up, as the newer driver for fast video card is well known to have problem (from the nvidia forum), so I never thought this is a motherboard issue until it hangs on the BIOS level at start up. If it happens again, I'll let you know the Q code.


----------



## squidman

Science,

Apart from cost, why the gtx 780 ti instead of say titan blacks? Iff it's for CUDA development? As far as I understand, the gtx 780's like a lot of the others, are crippled regarding cuda and floating-point performance (although evga is coming out with a 6gb version of the 780 if you can live with that and need vram for say cuda rendering). I would have thought for scientific applications the semi-pro titan would be much better than the gaming-centric 780's. And iff you're running scientific applications which are opencl-based, well then the lowliest amd card will thrash even the titan. Have you looked at benchmarks on your application of choice? Which is?

What an awesome build btw! Caselabs rocks, was just too expensive for me (using 900d) and wouldn't fit under the desk (have 1cm to spare!). Would imagine you really have to sort your airflow and positive/negative pressure to keep all those gpu's cool iff you're not watercooling!

Yeah agree this board and this kind of extreme build with usually minimal support is pretty scary! Thank god for the forums is all I can say! I guess it comes with the territory when you go ba11s out on a build!







Good luck and keep us posted!

Blademaster is the man, all-knowing guru on this thread at least! All bow down!

Edit: need an html editor here to be able to embed proper smileys admin!


----------



## TheBlademaster01

Quote:


> Originally Posted by *happy hopping*
> 
> the xeon is E5-2609, I'll dig out the BIOS Ver. later, for now, it should be the latest, the Oct. 2013 ver., I'll let you know. As to that cursor hang problem, that is back in Jan., I didn't check the Q code, I should have, see at the time, I thought it's a nvidia driver acting up, as the newer driver for fast video card is well known to have problem (from the nvidia forum), so I never thought this is a motherboard issue until it hangs on the BIOS level at start up. If it happens again, I'll let you know the Q code.


You don't really need to check the BIOS version if those are retail. It might be a defective motherboard, or defective keyboard. Does the keyboard work on other computers (correctly)?

It could never hurt to flash the BIOS to the latest version though.
Quote:


> Originally Posted by *squidman*
> 
> Science,
> 
> Apart from cost, why the gtx 780 ti instead of say titan blacks? Iff it's for CUDA development? As far as I understand, the gtx 780's like a lot of the others, are crippled regarding cuda and floating-point performance (although evga is coming out with a 6gb version of the 780 if you can live with that and need vram for say cuda rendering). I would have thought for scientific applications the semi-pro titan would be much better than the gaming-centric 780's. And iff you're running scientific applications which are opencl-based, well then the lowliest amd card will thrash even the titan. Have you looked at benchmarks on your application of choice? Which is?
> 
> What an awesome build btw! Caselabs rocks, was just too expensive for me (using 900d) and wouldn't fit under the desk (have 1cm to spare!). Would imagine you really have to sort your airflow and positive/negative pressure to keep all those gpu's cool iff you're not watercooling!
> 
> Yeah agree this board and this kind of extreme build with usually minimal support is pretty scary! Thank god for the forums is all I can say! I guess it comes with the territory when you go ba11s out on a build!
> 
> 
> 
> 
> 
> 
> 
> Good luck and keep us posted!
> 
> Blademaster is the man, all-knowing guru on this thread at least! All bow down!
> 
> Edit: need an html editor here to be able to embed proper smileys admin!


Hahaha, thanks for the compliment but I still don't know half the functions of this board (mainly storage)


----------



## happy hopping

In the asus website, it now has a few new BIOS. But it's bug fix for:

1. Improve the FAN curve settings.
2. Patch the compatibility for K4000.

1. Support Windows 2012 R2.
2. Improve compatibility for add-on VGAs while 2 CPUs installed.
3. Improve compatibility for add-on VGAs with Ivy-Bridge CPUs.

Regardless, IF I do flash it, then does that mean I no longer need to clear my CMOS?

the keyboard is "occasional" failure, I'm not convinced it's defective, as when I connect the USB flash drive on the USB 3.0 port , the PC should response, so it could be the motherboard, but I'm hoping it isn't


----------



## Morachi

Hi folks,

It has been a good year since I posted here - back when the Z9 wouldn't play nice with the K5000 GPU. I ended up selling it as Asus took another 6 months to get the issue straightened out and I simply couldn't afford to have such expensive hardware laying about not doing anything. So the poor rig ended up having an old GTX 480 put in place.

Whilst i'm not looking to go for another Quadro, my other half has been eyeing off a new GPU for general duties. I've been looking at the GTX 770, and wanted to know how the 7 series cards handle with this motherboard - keeping in mind I don't fancy going to nuts with the BIOS updates in case the microcode doesn't support the Corsair RAM or ES CPUs I have in place presently.

Anyone with non-ecc ram and perhaps ES CPUs with a 7 series card in place, id love to hear about your config so I don't go throwing cash at a card that may not work with this tedious motherboard.

Cheers!
Dave


----------



## TheBlademaster01

Quote:


> Originally Posted by *happy hopping*
> 
> In the asus website, it now has a few new BIOS. But it's bug fix for:
> 
> 1. Improve the FAN curve settings.
> 2. Patch the compatibility for K4000.
> 
> 1. Support Windows 2012 R2.
> 2. Improve compatibility for add-on VGAs while 2 CPUs installed.
> 3. Improve compatibility for add-on VGAs with Ivy-Bridge CPUs.
> 
> Regardless, IF I do flash it, then does that mean I no longer need to clear my CMOS?
> 
> the keyboard is "occasional" failure, I'm not convinced it's defective, as when I connect the USB flash drive on the USB 3.0 port , the PC should response, so it could be the motherboard, but I'm hoping it isn't


I noticed that while flashing the BIOS resets BIOS settings, you sometimes need to remove the battery to truly reset the board (something about the management engine?). I'd flash and remove the battery after it is completed. Make sure that it completed flashing though, you might ruin your BIOS if you power down the system in the process of flashing (but I guess you already know that).
Quote:


> Originally Posted by *Morachi*
> 
> Hi folks,
> 
> It has been a good year since I posted here - back when the Z9 wouldn't play nice with the K5000 GPU. I ended up selling it as Asus took another 6 months to get the issue straightened out and I simply couldn't afford to have such expensive hardware laying about not doing anything. So the poor rig ended up having an old GTX 480 put in place.
> 
> Whilst i'm not looking to go for another Quadro, my other half has been eyeing off a new GPU for general duties. I've been looking at the GTX 770, and wanted to know how the 7 series cards handle with this motherboard - keeping in mind I don't fancy going to nuts with the BIOS updates in case the microcode doesn't support the Corsair RAM or ES CPUs I have in place presently.
> 
> Anyone with non-ecc ram and perhaps ES CPUs with a 7 series card in place, id love to hear about your config so I don't go throwing cash at a card that may not work with this tedious motherboard.
> 
> Cheers!
> Dave


Okay this is the deal







. I see you have QA91 chips which means you are pretty much in the same situation as I am. The GTX 770 will have the same issues as your K5000 had.

You can only use PCI-E 2.0 cards (Fermi/Northern Islands) reliably on older BIOS versions (pre 3109). Your chips won't post on version 5004 and I read here that someone was able to post with B0 chips on version 5206 (not tested personally).

If you want to use PCI-E 3.0 cards you need SNB-EP C0/C1 (QBxx) or IVB-EP (QDxx/QExx/QFxx) or of course any retail chip on BIOS version 3506 and later.

I have QA8Z chips on BIOS version 3302 and a GTX 780Ti and it's a big headache but I can't upgrade atm (funds







). 3206 is crap as well with these chips (doesn't work properly with either the 780Ti or 470), 3109 worked with my GTX 470 but I can't flash back, 0703 worked with 470 (780Ti won't post), 0503 same as 0703. 3506 works somewhat better than 3302 but runs my RAM in Triple Channel rather than Quad (sees the full 32GB of RAM though)

As for the BIOS version I am using (3302). It locks up in Windows sometimes and for some reason can't install OSes to HDDs not connected to the Intel SATA controller. Ubuntu works a lot better but my GTX 780Ti is called GeForce Ashley







, which causes a whole lot of different problems. Also, I only get display (once in like 3 cold boots) via the HDMI output of my 780Ti installed in PCI-E slot 3 (second blue slot from the top). DVI ports give distorted image and lose signal when booting into all OSes (tested Windows 7, OpenSUSE 12.3, Ubuntu 12.04).

The strangest issues really, though I have a theory as to what is wrong but nothing solid (with proof).


----------



## Morachi

Interesting, as the issue with the K5000 was actually the 256mb Memory BAR when registering at POST. Which is why the card (K5000) would boot 1 in 4 times - when it got into Windows, all was fine. My GTX 690 works fine, but it only uses a 128mb Memory BAR in the address registers. I actually proved this was the issue (the firmware was badly written on the Z9 - hell there was a primer for exactly how to fix this, as it was a known issue from 2006), via talking with a PNY engineer who sent me a test firmware for the K5000 to lower the BAR to 128mb - worked flawlessly every time, albeit at the price of performance.

Interesting if the 7 series has raised the Memory BAR to 256mb, in which case, Asus still haven't properly solved the issue for the K5000, or are using a band aid fix.

EDIT: The issue was the Z9 couldn't allocate enough resources to satisfy the 256mb Memory BAR requirements of the GPU, just for clarification.


----------



## squidman

I have a zotax gtx 770 4gb vram board, and it won't post on the (blue) pcie v3 slots (reliably) ...just on the black ones...am on bios 3506 according to sisoft sandra, but think am on later (srry give me a few minutes, testing new headphone amps!)...never did reliably post on the true gen-3 pcie slots...back in 10...

uhhmm...little help here? how to check bios version without rebooting? machine takes 15 minutes to start up again? Pretty sure I was on 5xxx....

Srry, testing akg k612's with headphone amp musical fidelity v-can headphone amplifier...well the DAC on the board is supposed to be good enough, and was with my old akg studio k141's...also have an older m-audio black box..

These newer cans just suck! Srry off topic! But you have been warned! Iff you have a great set of cans, keep them and resolder! Gotta dig out soldering iron again...


----------



## TheBlademaster01

My GTX 470 also doesn't post anymore after flashing to .CAP bioses.

And I doubt it is the BAR addressing for BIOS versions past 3506







. After that it really comes down to Sandy B0 messing up the Intel Management Engine.

My system was fine in Windows as well except when running it for more than a week under 100% load or so. I noticed that only stressing the CPUs or GPUs didn't result in a lockup. Stressing them both often resulted in a lock up (or reboot with a different Q code displayed on the LED).

In Ubuntu it's the same patter but less often and it just locks up and never reboots.

Also, Radeon 7xxx and later don't work on Sandy B0 either.

I should note that my 780Ti works at PCI-E 2.0 speeds at all times.

My system was stable for a month 24/7 100% load with my GTX 470 and BIOS 0503 though.

What BIOS version were you running with the 690?


----------



## squidman

Am on e5-2665 B0 stepping qa8x (I think but at least am sure of b0)...will have to reboot to check bios...wow I miss my old cans! And admin can you PLEASE allow insert of proper smileys?


----------



## TheBlademaster01

Sorry, you ninja'd me









HWInfo64 shows BIOS version.


----------



## squidman

Well on reboot am definitely on 3506...just not messing with it anymore with the raid5 and all...putting the 770 in a pci-e 3.0 blue slot never worked for me.

Blademaster! You are the friggin ninja! wish I had known about that program before! But installed on the laptop here, so have to wait for friggin' beast to reboot...verry cool indeed! But how come you know so little about html?


----------



## Science

Quote:


> Originally Posted by *squidman*
> 
> (...)
> Apart from cost, why the gtx 780 ti instead of say titan blacks? Iff it's for CUDA development? As far as I understand, the gtx 780's like a lot of the others, are crippled regarding cuda and floating-point performance (although evga is coming out with a 6gb version of the 780 if you can live with that and need vram for say cuda rendering). I would have thought for scientific applications the semi-pro titan would be much better than the gaming-centric 780's.


Thankfully for my software double precision is not the case. Software I use doesn't employ dp calculations







I am happy, because I don't need Tesla and Titan! It will be a huge money lost if I would buy Tesla or even Titan. So, I am happy
Quote:


> And iff you're running scientific applications which are opencl-based, well then the lowliest amd card will thrash even the titan.


For me, the best way would be to get rid of CUDA and have all software in OpenCL, but it is not possible yet. Only one program is using OpenCL







In the future I hope all software I need will be OpenCL compatible.
Quote:


> Have you looked at benchmarks on your application of choice? Which is?


there are no benchmarks for bioinformatics software







I performed all of them by myself








Quote:


> What an awesome build btw! Caselabs rocks, was just too expensive for me (using 900d) and wouldn't fit under the desk (have 1cm to spare!).


Thanks - but, still, it is only on paper







There is a huge chance I will get all stuff by the end of April.
You can always use smaller CaseLabs: Magnum two-sided.

Quote:


> Would imagine you really have to sort your airflow and positive/negative pressure to keep all those gpu's cool iff you're not watercooling!


I promise, I will post pictures when air and later water cooling will be applied to CaseLabs STH10. IMO both will be just great.
Quote:


> Yeah agree this board and this kind of extreme build with usually minimal support is pretty scary! Thank god for the forums is all I can say! I guess it comes with the territory when you go ba11s out on a build!
> 
> 
> 
> 
> 
> 
> 
> Good luck and keep us posted!
> 
> Blademaster is the man, all-knowing guru on this thread at least! All bow down!


agree and again thanks.Sorry for savings in words, but my space key is working very hard, and it is very difficult to write.


----------



## Science

I am a little big shocked.... I found E5-2680v2 ES QF6T for ..... !!! 717 Euro a piece !!!! Although I would like to buy a retail version of 2697v2 (2306 Euro a piece







) *I can SAVE A LOT OF MONEY!*!.. But I don't know if these ES would be good for this motherboard... I am confused! Risk or not.... ??? A guy who is selling these ES would be able to find 2697v2 for me later







I am confused! I am confused... Please help.


----------



## TheBlademaster01

QF6T are QS chips. They should be the closest thing to retail. Pretty darn good deal IMO


----------



## Science

Quote:


> Originally Posted by *TheBlademaster01*
> 
> QF6T are QS chips. They should be the closest thing to retail. Pretty darn good deal IMO


so..... Should I buy ES E5-2680v2 chips or retail E5-2697v2??? This guy gives 3 months of *written guarantee* for these ES chips... it is better than nothing IMO.

Then, later he would find E5-2697v2 for me... If these ES chips are the same as retail or almost the same.... I think it is a good idea to take the risk. But I am a noob in the world of Xeons and ES


----------



## TheBlademaster01

I would take the risk. Especially if he allows you to return it if it doesn't work. QF6T should work though.


----------



## squidman

I would have risked ES chips (it's only a few mhz? have to look up...agree 12 cores drool! - haven't seen any es 12-core tho')...after all hmmm what can you buy for 3200 euro`? Say 4x titan, or the new titan z, or a a complete xeon rendering-node if that's what you use it for, or a 4k cinema camera, or at least three vacations somewhere in europe, or a holiday to australia......

Much monies, so possibilities!









Iff it's for work though, and you can get a tax write-off...well the retail (should!) be more stable....iff your cash-flow can afford it...dunno iff it makes a difference with this temperamental board all things considered...


----------



## squidman

Science...

Check these out (only 2.1 ghz, but they are 12-core)....http://www.ebay.com/itm/intel-E5-12cores-2-1Ghz-Xeon-E5-2695-v2-2687W-2667-V2-ES-engineering-sample-/261333817274?pt=US_Server_CPUs_Processors&hash=item3cd8b597ba

848 usd for genuine 12-cores......gives you an extra 4 cores over the 10-cores.....note they are an earlier B1 stepping which could be problematic with bios....man if I had the dough.....they get 13.5 on cinebench each whereas BOTH my e5-2665's get 17.8.....


----------



## TheBlademaster01

QF6T should get over the 14 per chip









Nothing wrong with B1 Ivy chips, but 2.1GHz is crippled pretty much. Retail 2695v2 would do over 16 points each. SR1BA are E5-2695V2 ES chips you'd want (2.3-2.4GHz) and do 15.5 points each


----------



## squidman

Guru! Crippled and crippled (3200 euro a pair cheaper crippled!)....

Question...running prime 95 and checking hwinfo64, with my piece of s...t corsair link running all maxed...cpu temps are very nice, around 44C at maximum (bit loud tho!), but further down, under sensors, the memory ambient shows 50c, which is fine, but the memory channels show 80c! That has me wondering whether I should stick in a spot fan (one of those bendy things). But does that relate to the memory cards themselves, or the heatsinks above the memory? Have 4 temp sensors plugged into the corsairlink pos





















, but usually only one or two work iff at all. Or maybe I should invest in some heatsinks for the memory (the cheapest corsair 1600 I could find) and somehow plop a fan in there (would need to be 2 on either side....http://www.amazon.com/Antec-Spot-Cool-SpotCool-System/dp/B000I5KSNQ

CPU power package (THAT may be the two silver heatsinks up the top) are reading 72 as well......

Don't need the h80i's of course but one nice thing is that they allow for this kind of thing as opposed to some giant noctuas blocking all airflows.

Addendum...quite a few peps have mentioned it might be an idea to reverse the rear fan (to stop dust accumulation and improve positive pressure), wondering whether it is worth it? Have at the moment going in, 3x120's that came with my 900d, 1x140 noctua on the bottom pulling air, and then of course the rear corsair 120 exhausting, the push/pull of the 4x120's from the h80i's exhausting out the top + one extra noctua 140 exhausting out the top. Gotta say, after 6 months looking pretty dusy in there. Too bad Corsair didn't provide dust filters at the back and on the bottom.

As usual, great forum and especially great thread! All hail the masters of the pc universe!!







(well that is us obviously, with the sickest possible builds!)....

Edit: with prime95 running, my B0 e5-2665's run at around 2400....but for me threads is more important than MHZ....wow still would love 2x12's!!!! May have to upgrade in the fall when I go back to school doing 3d animation...8 more threads...drool....1700 bucks for them ES version...(what to do with the old ones?)...


----------



## TheBlademaster01

Not really an expert on airflow (been a long time since I built an air cooled build







)

RAM shouldn't matter though. CPUs are unreasonably high though. They should run really cool, but 72*C is not a problem.


----------



## squidman

CPUs are running cool....43C max, but the MEMORY CHANNEL is running at 80c, which I assume is the memory dimms. Or the two silver heatsinks above the cpu....(assuming they are cooling the qpi links from the memory to the cpu's)...

BTW my Bios 3506 is a .cap bios, but don't think earlier versions helped with the pci-e slots....nobody has got earlier s-b b0/b1 etc to work with later versions? Not sure iff v.2 to v.3 of pci-e makes such a huge difference....


----------



## squidman

You mean these as far as e5-2695? http://www.ebay.com/itm/INTEL-XEON-12-CORE-PROCESSOR-E5-2695V2-2-40GHZ-30MB-CACHE-8-GT-S-QPI-115W-SR1BA-/310909647378?pt=LH_DefaultDomain_0&hash=item4863a8ba12

USD 2290 vs 890....!!!







for 3ghz I can go make an extra cup of coffee....

or at 2k usd same 2.4ghz version...http://www.ebay.com/itm/Intel-Xeon-Processor-E5-2695-v2-30M-Cache-2-40-GHz-/131051577634?pt=CPUs&hash=item1e83486122

still pricey....why not get 4! of the 'crippled' e5-2695's at 2.1 ghz and build a node with change left over? I.e. 48 cores 96 threads at 2.1 ghz vs 24 cores/48 threads at 2.4 ghz....(ok have ...to...keep...visa...in...wallet....







)..


----------



## TheBlademaster01

I mean this one.

http://item.taobao.com/item.htm?id=38115995082

And no, PCI-E 2.0/3.0 doesn't matter on .cap since they all kind of fail to post at times. .rom doesn't work with 3.0 cards so that's why you can reliably use 2.0 cards (if you use .rom bioses).

You should really only use B0 chips on .rom BIOSes.


----------



## squidman

Well reverting means ordering new bios chips, and it seems to be working....

Regarding temps, will try and upload some here...but seem to recall hwinfo gave out some rubbish numbers where there weren't sensors or something....prbly when I first started build....


Mind you, running prime95 for 30 mins....at least the cpu's stay nice and frosty! Had this issue with sisoft sandra. May still wack in some spot fans and extra noctuas on the bottom for intake.....for when I'm rendering 24/7 or out of town and leaving it on for teamviewer...don't really want it burning down the flat....


----------



## squidman

Dude! what a find! Ok forget fleabay.......too bad I just spent available cash on skiing gear!







(but I love my phat skis!) I take it that's in yuan...(or 627 usd?)....


----------



## TheBlademaster01

I haven't ever checked those sensors on my system tbh









And yes, those are at yuan.

Here's an English taobao agent for the ad

http://www.bhiner.com/index.php?mode=Search&skey=http%3A%2F%2Fitem.taobao.com%2Fitem.htm%3Fid%3D38115995082&search_eng.x=-671&search_eng.y=-15&search_eng=search


----------



## squidman

Cool! Thanks for the tip! Stay frosty (no pun intended...)...you're still the







in my book!


----------



## TheBlademaster01

Thanks man, whatever that may mean


----------



## squidman

あなたは忍者だ!!!! Will save you the time...you're still ninja.....

Running prime95 with stupid corsair link (don't get me started!) on lower values, cpu temps were low (46+47C) but then abrubt crash. This happened running prime95 with everything on full blast, it still crashed but lasted an hour or so. Does not bode well for 24/7 rendering (albeit rendering even on vray or mental ray, cpu's are rarely pegged!)...still a bit concerned.

Ok have to log off and somehow get up early enough for the skibuss!









Oohh great katana master, my cpu's are on watercooling (h80i's) but the rest is unfortunately air-cooling...I have too much shame, may I be allowed to perform ritual (切腹 seppuku) by adding further noctuas or a rad or two? (preferably a noctua or two in the bottom!)...


----------



## TheBlademaster01

Hahaha, I actually learned how to read hiragana and kanji









Happy skiing man, also don't get too drunk


----------



## squidman

Too late!









But I should be able to get up in the am...I hope (on IT/reverse normal people time here with deadlines)...damn 6 am start....no genever, just a couple/4/6 brews...like other peps in the business, going to bed time is around 2-4am or something...not quite like Best NL in my younger days or Amsterdam NYE...pretty crazy!


----------



## Jann

I'm thinking of switching my Xigmatek Elysium to something smaller and if possible - quieter.
From Tek Syndicate video I know for a fact that our mobo can be installed into Define R4.
Judging by sizing charts, it seems that Corsair Air 540 should be able to fit it too.

Now this is where I can't decide which one to pick since I can get both for the same price.
I know R4 should be quieter, but how good will the cooling be. I do plan to add/swap case fans at some point but for a while system would be running with stock fans only.

Anyone with any experience with these two cases? Or with any smaller form factor cases and the Z9PE-D8.


----------



## halkx

OK, long time lurker but first time poster in here...usually hang out at 2cpu...

I've been given a brand new Z9PE-D8 and I am looking forward in building a dual-cpu rig after quite a while...my last one was a SR2 which I enjoyed very much, both building and using...now with this new beast...My last step towards building the system is buying the cpus for it...

I found these

http://www.ebay.com/itm/QTY-1x-Intel-CPU-E5-2648L-8-Cores-1-8Ghz-20MB-Cache-LGA2011-ES-QB7Z-/141241908992?pt=CPUs&hash=item20e2ac7f00

and

http://www.ebay.com/itm/QTY-1x-Intel-CPU-E5-2648L-8-Cores-1-8Ghz-20MB-Cache-LGA2011-ES-QBXU-C2/141188297562?_trksid=p2046732.m2060&_trkparms=aid%3D111000%26algo%3DREC.CURRENT%26ao%3D1%26asc%3D27%26meid%3D5901746153548086680%26pid%3D100040%26prg%3D1011%26rk%3D4%26rkt%3D4%26sd%3D141241908992%26

that are close to my price range (up to 250-300 eur per cpu)

I want to be absolutely sure that the cpu that I get will work with the current bios version the board has (which I suspect is a .rom version be able to work with the.cap versions as well until the latest one)...

what do you guys think? any other suggestion for cpus will be most welcome

Thanks a lot

Michael


----------



## TheBlademaster01

QB7Z are fine with any revision









Lucky you for receiving this board









@jann

I know of one person who has the Air 540, but not in combination with the Z9. Can't really help with that.


----------



## halkx

Quote:


> Originally Posted by *TheBlademaster01*
> 
> QB7Z are fine with any revision
> 
> 
> 
> 
> 
> 
> 
> 
> 
> Lucky you for receiving this board
> 
> 
> 
> 
> 
> 
> 
> 
> 
> @jann
> 
> I know of one person who has the Air 540, but not in combination with the Z9. Can't really help with that.


Blade thanks for the reassurance...thought so too by myself...do you think there is going to be any difference between QB7Z and QBXU that justifies the price diff or should I go for the cheaper one...

Yeah, I was lucky that a friend of mine wanted my S1155 motherboard in exchange for this...


----------



## TheBlademaster01

I don't think so. Could be a revision difference. QB7Z is C0, I'm not sure about the QBXU but it should be C stepping. All are E5-2650L QS though (also called E5-2648L by some).


----------



## halkx

Well, from what the seller says in the listing QBXU is C2 stepping...which is the same as the retail stepping if I am not mistaken...


----------



## TheBlademaster01

C1/C2 are retail equivalents indeed (usually sent to reviewers etc.)


----------



## 82ghost82

Stupid question: is there a way to check if a new bios release will be compatible with an engineering sample?


----------



## halkx

Well, from what I understand, all you have to do is make sure you have a C0 chip at least (QBXX sSpec for SB xeons). Until now all bios updates are compatible with them, and they have far fewer incompatibilities with gpus etc. the v2 ivy xeons are compatible with the newest versions of bios.
This of course does not guarantee compatibility with future versions but my feeling is that, since no new chips will be available for this socket, there is no reason For asus to break compatibility with older ones...


----------



## TheBlademaster01

What halkx says.

C0 and later are compatible with all BIOS versions (Sandy Chips that is)
Ivy ES is compatible with 5004 and later, but 5004 requires ECC RAM in combination with Ivy (even retail IIRC).
Sandy B0 and A1(?) are compatible with 0503 - 3109 without issues and 3206 - 3506 also "works" with Sandy B0, but has VGA incompatibilities and other bugs with Sandy B0.

I'm going to make a compatibility chart I think


----------



## halkx




----------



## 82ghost82

Thanks for your replies; yes I have a c0 chip qb7r but what I want to know is if you can for example extract microcodes from the bios and see if they're compatible with the chip.

Theblademaster: I know you are expert and if you state what you write then I know it's true, but how can you say a bios is compatible with c0?


----------



## TheBlademaster01

From experience and what I have seen and heard (with proof or reliable sources)









I hope this helps


----------



## TheBlademaster01

Quote:


> Originally Posted by *82ghost82*
> 
> Thanks for your replies; yes I have a c0 chip qb7r but what I want to know is if you can for example extract microcodes from the bios and see if they're compatible with the chip.
> 
> Theblademaster: I know you are expert and if you state what you write then I know it's true, but how can you say a bios is compatible with c0?


There is no way to verify AFAIK. You just need to observe and I guess trust other people's findings. C stepping CPUs are pretty much equal to retail samples so they shouldn't really have those bugs. B stepping or even A stepping CPUs are a completely different story.


----------



## 82ghost82

Thanks!the chart helps a lot








I confirm no problems at all with 5404 and 2x qb7r


----------



## TheBlademaster01

Z9PE-D8WSCompatibility.pdf 73k .pdf file


Here's a PDF in case you can't read it 

E:

Yes, QB7R are pretty much retail E5-2687W. You should however make sure 5404 works with the other peripherals. Asus is known to brick things in the process (think storage, VGA etc.). I have no data on that.


----------



## squidman

Phxtriode,

Iff you're getting dual H80i's, you may need to pick up some nzxt usb expanders....I never got mine to work via the corsair link garbage reliably unless they were hooked up to internal usb headers
(tried hooking to external usb, no dice). The board doesn't have enough internal usb 2 headers....ended up getting two http://www.nzxt.com/product/detail/99-iu01-usb-expansion-case-accessory...genius bit of kit...









The fans on the h80i's are friggin' noisy, prbly worth replacing with noctuas....on the other hand, my temps even with fans at very low never go above 50c, at which speed they are bearable (but not silent!). Even running everest or prime95, it's enough to have them on a medium setting, my e5-2665's never go above 60c. Your xeons may run hotter of course...


----------



## squidman

Upgrade advice needed....on e5-2665 B0 QA8X stepping, considering upgrading to B1 E5-2695 v2 or E5-2697 v2 (on Taobao for around a grand). Will I need to move to ECC memory for that upgrade, and what are the limitations regarding BIOS? Looks like someone even managed to OC some 2697's (not too many details, can't find link...top cinebench 11.5 scores, had around 34 with bclk of 104? and multiplier of 34? iff I remember)...no mention of stepping. Even a score of 32 is a huge leap from my current scores of 17.8. Will be doing a lot of rendering this autumn, taking 3d courses.

Sorry! Too pressed for time to search the forums this time around....deadlines...







May your cpu's stay frosty....

Edit: never mind, thanks to Blademaster and his chart all is revealed....


----------



## 82ghost82

Quote:


> Originally Posted by *squidman*
> 
> Phxtriode,
> 
> Iff you're getting dual H80i's, you may need to pick up some nzxt usb expanders....I never got mine to work via the corsair link garbage reliably unless they were hooked up to internal usb headers
> (tried hooking to external usb, no dice). The board doesn't have enough internal usb 2 headers....ended up getting two http://www.nzxt.com/product/detail/99-iu01-usb-expansion-case-accessory...genius bit of kit...
> 
> 
> 
> 
> 
> 
> 
> 
> 
> The fans on the h80i's are friggin' noisy, prbly worth replacing with noctuas....on the other hand, my temps even with fans at very low never go above 50c, at which speed they are bearable (but not silent!). Even running everest or prime95, it's enough to have them on a medium setting, my e5-2665's never go above 60c. Your xeons may run hotter of course...


Hi have also 2x corsair h80i: firstly I connected them to the internal usb header, but that was not very good as I couldn't have usb type A ports in the front of the case; so I bought from amazon 2x usb cables with 90 degree mini usb and I coneected them to type A ports at the back of the case.
No problem at all with corsair link for both solutions.

Also with 100% cpus usage (2x qb7r, 150W tpd each) running scientific calculations, with all 4 fans at maximum speed I never go above 50 ˚C - 55 ˚C.


----------



## halkx

How about noise at full speed?


----------



## VitOne

This is my RIG after months of hardware changes. I already installed 8 RAM modules (not show in pictures) but I may need to change them. I don't need ECC and I would like to know what kind of memory you would suggest (total 32 or 64 gigs, I am checking RAM usage with some simulation softwares and I may not need 64).


----------



## ocvn

Quote:


> Originally Posted by *squidman*
> 
> Upgrade advice needed....on e5-2665 B0 QA8X stepping, considering upgrading to B1 E5-2695 v2 or E5-2697 v2 (on Taobao for around a grand). Will I need to move to ECC memory for that upgrade, and what are the limitations regarding BIOS? Looks like someone even managed to OC some 2697's (not too many details, can't find link...top cinebench 11.5 scores, had around 34 with bclk of 104? and multiplier of 34? iff I remember)...no mention of stepping. Even a score of 32 is a huge leap from my current scores of 17.8. Will be doing a lot of rendering this autumn, taking 3d courses.
> 
> Sorry! Too pressed for time to search the forums this time around....deadlines...
> 
> 
> 
> 
> 
> 
> 
> May your cpu's stay frosty....
> 
> Edit: never mind, thanks to Blademaster and his chart all is revealed....


You can't do 34, i tried all bios and max for 2697v2 is 30, however, i managed to 110


----------



## artzgo

I recently built my computer on the CPU E5-2697 V2.
So far, there is one CPU, as it will be more money to buy an other.
At the beginning of the computer run poorly, there were considerable delays.
After I set up the BIOS and now it is OK
Include photos of the BIOS may be useful to someone.
If anyone has an idea of the setting, would like to use.


----------



## 82ghost82

Quote:


> Originally Posted by *halkx*
> 
> How about noise at full speed?


It's very very noisy, but for me it's not a problem as the workstation is in another room and when it calculates also at night I don't hear the noise.


----------



## halkx

Quote:


> Originally Posted by *82ghost82*
> 
> It's very very noisy, but for me it's not a problem as the workstation is in another room and when it calculates also at night I don't hear the noise.


So I will have to find a way to lower the fans rpm (through ilink maybe?) or change the fans to quieter ones with the same efficiency? Recommendations anyone? Slightly off topic I know but...


----------



## 82ghost82

Quote:


> Originally Posted by *halkx*
> 
> So I will have to find a way to lower the fans rpm (through ilink maybe?) or change the fans to quieter ones with the same efficiency? Recommendations anyone? Slightly off topic I know but...


OT, sorry









Well, if you change the fan profile through corsair link from full speed to balanced mode or quite mode the noise is very acceptable; I don't think there will be much difference in fan brands about noise (however I have never tried to change the fans).
Temperatures with balanced/quite mode will be acceptable and they will not hurt cpu(s).

Since I haven't problems with noise I prefere to have the lowest temperatures as possible, as when I run simulations all my 16 cores go to 100% usage, but sure you can low the rpm of the fans that cool the radiator(s) without any damage.

Remember to change bios settings to have cpu fan at "full speed": this will give full voltage to the pump(s).


----------



## halkx

Ok guys I need opinions

I am between two sets of CPUs to power this beast

A pair of 2670 C0 offered by a fellow member here at a very reasonable price and a pair of 2650L I found on the bay for 2/3 of the price of the 2670s...Cooling will be dual H80i and the beast is going to be used primarily as a gaming machine....which CPU should I choose?










Decisions...decisions...


----------



## TheBlademaster01

2670s easily.


----------



## DizZz

Quote:


> Originally Posted by *TheBlademaster01*
> 
> 2670s easily.


Agreed. They are the best performance/watt sandy bridge xeon CPU









EDIT: Why are you using dual xeons for a gaming build? There are so many better options and you won't see good performance at all compared with an overclocked single cpu setup.


----------



## lowfat

Ya a Z9PE-D8 WS isn't a great board for gaming. Since you cant overclock there is a very good chance you will CPU bottleneck in games.


----------



## halkx

Yeah I know...









It's just for the fun of it...I am not a hardcore gamer...my favorite game is diablo....and since I was given this board at a next to nothing price...


----------



## halkx

Quote:


> Originally Posted by *lowfat*
> 
> Ya a Z9PE-D8 WS isn't a great board for gaming. Since you cant overclock there is a very good chance you will CPU bottleneck in games.


So you think that the 2670s will be a bottleneck as well?


----------



## TheBlademaster01

Depends on the GPU setup. Your system would perform the same as a stock 2500K in games. I haven't gamed in a long time but I think you'll need some more horsepower for multi GPU setups.


----------



## halkx

Quote:


> Originally Posted by *TheBlademaster01*
> 
> Depends on the GPU setup. Your system would perform the same as a stock 2500K in games. I haven't gamed in a long time but I think you'll need some more horsepower for multi GPU setups.


booo...not even an i7???


----------



## TheBlademaster01

Well, as many cores as it scales to. An i7 performs pretty similar to an i5 in games.

2500K = 3.3GHz = max turbo of 2670 (low threaded load).

If it would scale across 8 cores or more it would be faster than a stock i7 of course.


----------



## halkx

well my games are not going to be very demanding...I plan to use a single 7970 for my gaming...


----------



## DizZz

Quote:


> Originally Posted by *halkx*
> 
> well my games are not going to be very demanding...I plan to use a single 7970 for my gaming...


You will see roughly the same performance as a stock 2600k in games.


----------



## squidman

Artzgo!!

thanks dude! looking at doing a taobao for the same (e5-2697 v2), after my last 4k render took 3.5 hrs!!!! on e5-2665....

What is your heat like? And stability? Are hynix modules from e.g. taobao reliable (what would you go, cheaper 16 gb ECC or a couple of huge but very expensive 1 module 32 GB ecc modules?). Vue Xstream just crashed my pc with 32gb, albeit with 20+billion polygons...first time I have EVER seen anything come close, not even after effects!

All advice on e5-2695/97 v2 appreciated, as that is likely the best we can do for this board...then it is lga 2011-3......


----------



## squidman

Quote:


> Originally Posted by *ocvn*
> 
> You can't do 34, i tried all bios and max for 2697v2 is 30, however, i managed to 110


Quote:


> Originally Posted by *ocvn*
> 
> You can't do 34, i tried all bios and max for 2697v2 is 30, however, i managed to 110


Quote:


> Originally Posted by *82ghost82*
> 
> OT, sorry
> 
> 
> 
> 
> 
> 
> 
> 
> 
> Well, if you change the fan profile through corsair link from full speed to balanced mode or quite mode the noise is very acceptable; I don't think there will be much difference in fan brands about noise (however I have never tried to change the fans).
> Temperatures with balanced/quite mode will be acceptable and they will not hurt cpu(s).
> 
> Since I haven't problems with noise I prefere to have the lowest temperatures as possible, as when I run simulations all my 16 cores go to 100% usage, but sure you can low the rpm of the fans that cool the radiator(s) without any damage.
> 
> Remember to change bios settings to have cpu fan at "full speed": this will give full voltage to the pump(s).


Even running hwinfo stress tests, my e5-2665's never got above 50c....using the corsair link software. However, the software is garbage...cocks up everytime you resize, doesn't recognize some sensors (cpu 1 in my case), crashes constantly etc. Shame, was an elegant solution for my h80i's...support is non-existant, you need to rely on forums.

Off-topic, but how do I get rid of my e5' - 2665 B0 here in Norway for a decent price iff I want to upgrade to 2697 v2's? Need a forum where people are too risk-averse to buying spicy from overseas or something..need to cover cost of 2x e5-2697 v2, + some ecc ram. Sorry 3.5 hours for 4k render is just killing me (1 frame!) and I can't afford amazon or other online ****..


----------



## artzgo

Quote:


> Originally Posted by *squidman*
> 
> Artzgo!!
> 
> thanks dude! looking at doing a taobao for the same (e5-2697 v2), after my last 4k render took 3.5 hrs!!!! on e5-2665....
> 
> What is your heat like? And stability? Are hynix modules from e.g. taobao reliable (what would you go, cheaper 16 gb ECC or a couple of huge but very expensive 1 module 32 GB ecc modules?). Vue Xstream just crashed my pc with 32gb, albeit with 20+billion polygons...first time I have EVER seen anything come close, not even after effects!
> 
> All advice on e5-2695/97 v2 appreciated, as that is likely the best we can do for this board...then it is lga 2011-3......


I use the computer to 3D, mainly LW.

CPU is cool, max 61C, for middle-class cooling air http://www.silentiumpc.com/fortis-he1225/
It's pretty quiet on the cooling, but I have a good and super silenced cover http://www.fractal-design.com/home/product/cases/define-series/define-xl-r2-black-pearl

The computer is stable as a rock. I use ordinary Non ECC memory.
How far is 32GB but in the future as I want to buy a second CPU to extend the second 32GB.
if I can help, I offer my support


----------



## squidman

Artzgo, are you using retail or QS or ES chips? Somewhat confused at whether say QS chips (as opposed b0, b1) need ECC memory which would cost as much as my pc build (at least in larger one stick versions).....trying to find a taobao deal on C0/C1 chips....my 2665's sandy are just too slow (wow never thought that would be a problem, but the better the rig, the better the render settings........









Edit: maybe a local ninja (blademaster or blackened tush?) can chime in here regarding which is the better buy, 2695 v2 or 2697 v2 regarding turbo throttling...pretty much the same price on taobao once you get to c0--c1 steppings (around 6800 yuan +)...(or a higher clocked 10 core? with less throttling?)...


----------



## RickSFO

Still having stability problems with B0 chips.









System locks up randomly. Not a BSOD.

Two 2687W B0
BIOS 0703
Win7 64
AMD Firepro 4900 (PCI-E 2.1)
GSKILL F3-12800CL9Q-16GBZL
Antec 1200 High Current Pro

Have tried Gskill recommended timings 9-9-9-24-2 and mobo defaults, with no difference.

When it locks up the error code on the mobo LED is sometimes 15, sometimes 1.

I'm getting a lot of "Overclocking Failed" when I reboot after crashes. Usually I can just press F1 to enter setup, go into BIOS, don't change anything, and just hit F10 to "Save and exit" and then it boots fine.

To try to fix the "Overclock failed error" I changed the "AI over clock tuner" to manual. Set to 100.

And I disabled the "Intel Speedstep". We'll see if that helps....

BIOS settings that may be relevant:
1) I have disabled TurboMode and will see if that helps.
2) Set VGA priority to offboard
3) Hyperthreading enabled

See attachments for other BIOS settings. (I disabled the Speedstep after I took these screen shots).

Thanks in advance! This is making me crazy...

Rick


----------



## squidman

CPU-Z is correct! Remember it reports DUAL channel memory, i.e. memory divided by half (reports 1600 as 800)...had same panic attack until I googled it. As far as bios, blade master and blackened tush are the gurus there, but on my B0 stepping have only gone to the first .cap bios, and there have been many nightmares updating from pre .cap bios...now I can't remember iff 0703 is .cap or not, but there was a definite upgrade path you had to follow....remember not possible to downgrade, iff you absolutely have to you order a new preprogrammed bios chip for 15 euro...ebay it.

Don't think you should disable turbo mode (as you can't OC, let the board turbo them until throttling sets in, whatever that may be (intel ark sheets), again ditto speedstep (same as turbo?) should prbly be on.

Just a newb here, ors know better..good luck! Srry deadline here in a few hrs, no time to look sh!!t up...


----------



## artzgo

squidman, This is a retail CPU, I know that it is very expensive but there are no problems in this configuration.


----------



## eyepiz

Quote:


> Originally Posted by *othersfollow*
> 
> im using 1866 ran here but R, ECC. it's worked fine so far.


Hello,

I just installed these same kits, well the 16 gig kit versions. Would you mind sharing your bios setting for the dram? crucial's website is pretty vague it just says CL13 1.5v


----------



## Miracl

Hello

Building 2 machines with this mainboard.

The ram that was ordered was Crucial Ballistix 1600mhz 2x4gb kits x4. Non ECC.
Cpu is 2x Intel Xeon E5-2620 v2.

The machine won't post at all with both CPU's installed. Get the Q-Code error "BC". One CPU works fine.
Is ECC ram required for two cpu's?

The other machine is not booting at all. I think the MB is DOA.


----------



## ZDngrfld

Quote:


> Originally Posted by *Miracl*
> 
> Hello
> 
> Building 2 machines with this mainboard.
> 
> The ram that was ordered was Crucial Ballistix 1600mhz 2x4gb kits x4. Non ECC.
> Cpu is 2x Intel Xeon E5-2620 v2.
> 
> The machine won't post at all with both CPU's installed. Get the Q-Code error "BC". One CPU works fine.
> Is ECC ram required for two cpu's?
> 
> The other machine is not booting at all. I think the MB is DOA.


Have you updated the BIOS? Sounds like you're running an earlier version that had issues with IVB-EP and Non-ECC memory.


----------



## TheBlademaster01

@squidman

You might want to look at 2.3-2.4GHz base clock E5-2695V2s. They should turbo up to 2.7-2.8GHz.

Next up are E5-2680V2s that turbo to 3.0GHz

@Miracl

Update the BIOS of both to at least 5206. They should post with regular RAM.

5004 doesn't work without ECC and disabling Isoc with dual Ivy CPUs (IIRC).

Ivy CPUs also don't post on BIOS versions before 5004. You can order BIOS chips pre flashed to 5206 or use a Sandy chip to flash the BIOS.

@RickSFO

Try 0503. It worked great for my B0 chips. 0703 is .ROM so you should be able to flash back. Do not flash to 3109 and later however!


----------



## Miracl

The mainboard has the 5403 bios, 2x cpu's give a BC error and no boot.

I'll look in to it tomorrow. They have a similar machine somwhere else at work with the same setup and that works, so this is strange.


----------



## TheBlademaster01

Can you try disabling Isoc in BIOS. And if it doesn't work try flashing to 5206.


----------



## Miracl

Ok, i'll try that tomorrow. The other machine won't boot at all. The MB seems DOA, so i'll check that as well.


----------



## Miracl

Couldn't find bios 5206, so tried with bios 5302.
Isoc is disabled in bios.

Still no boot with 2x cpu's. Get the "BC" error on the Q-Panel just as before.


----------



## Miracl

Swapped out all the ram with ECC memory and now everything works. So dual v2 cpu's require ecc memory it seems.


----------



## ZDngrfld

Quote:


> Originally Posted by *Miracl*
> 
> Swapped out all the ram with ECC memory and now everything works. So dual v2 cpu's require ecc memory it seems.


I'm running 2670 V2s with Non-ECC.


----------



## BLinux

hello... i'm considering building a workstation machine with this motherboard, so subscribing!

quick question, and sorry if this is somewhere in this thread, i'm only beginning to read it... does this motherboard support 4x PCI-E x16 with actual x16 lanes in all 4 slots? The info on ASUS website isn't clear to me... also, to get PCI-E 3.0 functionality requires a Ivy-bridge-E processor, correct or no?


----------



## Krobar

PCIE-3 can work with Sandy Xeon but is not guaranteed and somewhat rare with Nvidia (AMD has a fairly high success rate).

The Asus uses a bridge and is not true 16 lanes for each slot to the CPU. Board that offer 4 X16s direct to the CPUs are available but not suitable for 4 card SLI (Good for compute or virtualized use).


----------



## BLinux

Quote:


> Originally Posted by *Krobar*
> 
> PCIE-3 can work with Sandy Xeon but is not guaranteed and somewhat rare with Nvidia (AMD has a fairly high success rate).
> 
> The Asus uses a bridge and is not true 16 lanes for each slot to the CPU. Board that offer 4 X16s direct to the CPUs are available but not suitable for 4 card SLI (Good for compute or virtualized use).


what bridge chip is Asus using and how many?

what are the other boards you are talking about? I'd like to research them.


----------



## Krobar

Quote:


> Originally Posted by *BLinux*
> 
> what bridge chip is Asus using and how many?
> 
> what are the other boards you are talking about? I'd like to research them.


Apologies, have that wrong it is direct connect to one cpu only (2X16 + 1X8 or 5X8)

4X16 at full speed (Two to each CPU):
http://www.tyan.com/Motherboards_S7055_S7055WGM3NR
I repeat though that these boards that spread across 2 CPUs are not suitable for SLI and likely not good for crossfire (Unsure on crossfire front, haven't tried it). The Tyan also offers a better NIC (Way better for virtualization) and remote KVM (Both of these features are useful if it is a server you're after). Storage is also SAS capable and all from the Intel Southbridge which is preferable in most cases.

Also with the exception of the Supermicro DAX boards neither Supermicro or Tyan offer overclocking (You can force RAM speed though).

This one is an option if you need loads of RAM:
http://www.supermicro.nl/products/motherboard/Xeon/C600/X9DRE-TF_.cfm
Some Lian-lis can be easily modded to accommodate the massive size (Built a machine for my brother based on the board above).


----------



## Science

Hesus I am back.... I hadn't realized buying this motherboard will be so difficult. I have been waiting for it 3 weeks! More over, I am not sure I will get Kingston ECC RAM on time. I decided to buy these ES - I should have them in about few days. But they wont work together without ECC RAM







This is my longest build







I am exhausted ..... best regards


----------



## 82ghost82

Hi all my friends!
is it possibile to start the system without the graphic card?
Any particular setting into the bios?

I'm planning to connect with infiniband a second workstation to my first one for parallel computations, but I don't need a screen on the second pc and I was planning to administer it through remote desktop.

Thanks


----------



## squidman

One for those running ivy's (digging some righteous power of tower in the background here! hey ya'all!!)...

Sooo Ivy e5-2695's v2 and e5-2697 v2's are running around 5-7k yuan and up on taobao, and there are QS chips out there for about that price. Do the QS chips need ECC memory? As opposed to B0 or B1 ES? A 32 gb hynix ecc ram at 1600mhz runs around 2500-3500 yuan, and you'll need at least two, or 4x8gb ecc ram....

So it adds a lot of cost on upgrading from sandy...on the other hand, this upgrade for 2k usd pretty much doubles myrendering power (cpu only) from my puny 18000 cinebench 11.5 to around 30...

Hoping others out there with ES Ivy bridge can chip in here, regarding ram, timings etc. Am not even sure for rendering e5-2697 v2 is the best, as other chips will turbo higher overall (albeit they have fewer cores)...for say VRAY, mental ray etc. Going back to school this fall doing 3d stuff, so the more rendering power the better...octane et al are still in beta as far as I'm concerned.

Still sitting on my puny sandy B0 QA8X e5-2665's , but wow they rock compared to a 5 year old laptop!









edit: power of tower..yeah right! after a 7 hour exam, bit knackered, lol!


----------



## GendoIkari

Hi all! I am currently using four Nvidia 7000 series Gpu's (Very Old, I know), In addition to two GT(?) 640's, and an LSI Megaraid card with one drive attached in my system. I was wondering if anyone got four GTX 580's working in the system, Especially the EVGA Hydrocopper 2's? I am dreading putting them in and then ending up with a dead board =/


----------



## sagitarius2k

hallo all

i just bought 2tb x 3 wdc green, and 128 gb kingston.

i want to active marvell caching, then i read the manual, it said "For SSD caching setup, you can use more than one SSD and only one HDD" ( page 2-35 )

does it mean if i want to use 128 GB ssd + 2TBx3 WDC green, i cannot use this feature?

second, how do i found Asus SSD caching software? i onto asus website, and cannot found of 'em

thanks


----------



## SRPlus

Hi,

This is my current system :-

• Asus Z9PE-D8 WS MoBo.
• Dual Intel Xeon E5-2687W 3.1Ghz, 8GT/s QPI (16 cores / 32 threads)
• Dual Cooler Master Nepton 140XL AIO CPU Liquid coolers
• 64GB Hynix DDR3 1600Mhz Quad Channel Memory
• 3x Nvidia GTX Titan 6GB
• 256GB Samsung 830 6gbps Solid State Drive
• 300GB Western Digital VelociRaptor 10,000rpm HDD
• 2TB Seagate 6gbps HDD
• Enermax EPM1500EGT Platimax 1500W Power Supply
• LG Blue-Ray, Lightscribe, DVD Writer
• Logitech Performance MX cordless Mouse
• Logitech G110 Keyboard,
• Wacom Intuos 5 large graphics tablet
• 2x HP ZR24w 24" 1920x1200, S-IPS LCD Monitor's
• Xigmatek Elysium case. 5 Akasa viper fans + BitFenix 200mm SP PRO
• O/S Windows 7 Professional x64 (SP1)

I am wanting to add a fourth GTX Titan for 3D production rendering in Iray.

The system runs perfectly with no issues, however I am a little concerned about adding the fourth Titan.

When I upgraded my PSU from 1200W to 1500w for the third Titan, I Tested the system with all 3 Titans in iray, and it worked fine, but tested it with 3x Titans and all 32 cores and it tripped the PSU.

I was told before buying the PSU by Enermax that the correct way to configure the pci-e outputs for 3x high end GPU's is as follows

Pci-e Outputs

[ 1 ] [ 2 ] [ 3 ]
[ 4 ] [ 5 ] [ 6 ]

Pci-e [ 1 ] 8pin to GPU 1 8pin
Pci-e [ 1 ] 6pin to GPU 2 6pin
Pci-e [ 2 ] 8pin to GPU 2 8pin
Pci-e [ 2 ] 6pin to GPU 1 6pin
Pci-e [ 3 ] 8pin to GPU 3 8pin
Pci-e [ 6 ] 6pin to GPU 3 6pin

CPU 2 on Pci-e [5]

EZ-Plug to one of the SATA V6 outputs, and Drives on the Sata V4 outputs

I tried swapping around Pci-e outputs 3 & 6 and this cured the problem. Now the system can run all 32 cores and 3x GTX Titans without tripping the PSU.

I learned later that there is no point using CPUs with the Titans in iray, as it just bottlenecks the CPU's, and knocks off very little render time..

But because of the PSU tripping out the first time, I am now concerned as to whether or not this PSU can handle a fourth GTX Titan or if it was just a load balance problem. I have seen a few other systems with Asus Z9PE-D8 boards running x4 Titans, with either Enermax Platimax or MaxRevo 1500w PSU's so it should be possible.

My other concern is about gaming. I currently have a 3-way sli bridge that I turn on or off depending on if I am working or gaming.

I am not sure if this card and motherboard will do 4 way sli. I think Titans are only 3 way.

So will I have to use a 3 way sli bridge with the fourth card on its own? Not even sure if the system will boot like that. or will the system revert to 3 way sli with a 4 way connector if the system is not compatible? No idea how exactly it works.

Anyone got a similar system running 4 GPU's ok at full load without tripping the PSU, and know if the system can do 4 way sli?

I was told by Enermax Tech support that the correct way to configure 4 way is as follows

[ 1 ] [ 2 ] [ 3 ]
[ 4 ] [ 5 ] [ 6 ]

Pci-e [ 1 ] 8pin to GPU 1 8pin
Pci-e [ 1 ] 6pin to GPU 2 6pin
Pci-e [ 2 ] 8pin to GPU 2 8pin
Pci-e [ 2 ] 6pin to GPU 1 6pin
Pci-e [ 3 ] 8pin to GPU 3 8pin
Pci-e [ 3 ] 6pin to GPU 4 6pin
Pci-e [ 6 ] 8pin to GPU 4 8pin
Pci-e [ 6 ] 6pin to GPU 3 6pin

CPU 2 on Pci-e [5], EZ-Plug Sata V6

Can anyone confirm this is correct?

Many Thanks


----------



## cgworks

Hello

I am new to the forum...
First of all i would like to say hello to all members...Sooo Hi!!

the reason i am writing in this thread is because i also own an Asus Z9PE-D8 ws..Problem is that i have 2x Xeon es 2687w B0 step 2 "very early silicone chip...i think the previous model can not even recognise a second cpu in a two cpu based motherboard...just so you know how early my version is...don't get me wrong...the cpu's are ok in general...they have not given me me many problems other than that the system does not seem to be working as smoothly as i expected it to be...it feels sometimes like it lags or bottlenecks...that's not the main problem though...I am using this system rig for 3d animation and video editing ect ect...
Problem is when i realized that the render times in vray or mental ray don't scale as someone would expect...so..let's do the maths...

Vray 2.40v test scene (same goes for vray vesrion 3.00)
1. with 1x Cpu 4.50 minutes...
2. with 2x Cpu 3.20 minutes...instead of half time (assuming that with one cpu is 4:50 than it should be 2:25 not 3:20 )

In Mental ray it gets worse..it doesn't matter if i have 1x cpu or 2x cpu in mother board ....render time is the same....

So,the only thing i am asking is for someone with the same rig ( mobo and cpu model) but with final silicone xeon 2687w could please download the scene files and post the render times...
i need only with 2 xeons no need for you to go and unplug the second cpu

that would be a great help for me so that way i would understand if it is the xeon's problem because of early stepping edition or because of the software not scaling properly..

Fun fact is that in cinebench with 2x cpu's the results i get are along with other similar rigs i saw online correctly...same goes for benchwell from maxwell renderer...

rig setup
asus z95ed8 ws
2 x xeon 2687w es B0 step 2
64 gb of ecc ram (from the qvl memory list )
ram config = 4 dimms of 8gb per cpu
gpu nvidia quadro 6000
intel ssd for Os
OS: windows 7 64bit pro
psu 1200 watt corsair

Any help would be appreciated...

Here are both scenes in a we transfer link

http://we.tl/ehNtHFt8iH

Thank you again in the first place...


----------



## Jann

Quote:


> Originally Posted by *cgworks*
> 
> Problem is when i realized that the render times in vray or mental ray don't scale as someone would expect...so..let's do the maths...
> 
> Vray 2.40v test scene (same goes for vray vesrion 3.00)
> 1. with 1x Cpu 4.50 minutes...
> 2. with 2x Cpu 3.20 minutes...instead of half time (assuming that with one cpu is 4:50 than it should be 2:25 not 3:20 )
> 
> In Mental ray it gets worse..it doesn't matter if i have 1x cpu or 2x cpu in mother board ....render time is the same....


Hi there.
I don't have the same CPU's (mine are E5-2660 C0), but I can comment on my experience with Vray and rendering in general.

Vray scales badly, especially on simple scenes like you provided. That's why most archviz people were advised to get OC'd 6 core i7's rather than Xeons. When OC'd to 4.6GHz they can do some scenes just as fast as our dual Xeons. My system rendered your scene at similar time, but in heavy scenes it should be much slower than yours.

To really see the difference try Corona, Maxwell or other unbiased render engine (maybe Vray in Brute Force, not sure). Only running benchmark scenes made for home renderfarms on Vray, and using Corona did I get significantly better results than a single OC'd i7 6core.
For Vray, you can go here and just get the FARM test scene. Results are the green tables.
I hope your system is ok, and it's just this Vray thing


----------



## cgworks

thank you very much for the fast reply and the info you provided...although we would like to hear from someone with same cpu because we are interested in buying new ones...(final version of the current ones)

thank you very much once again!!


----------



## cgworks

guys any thoughts on the weird problem with the scaling on vray or mental ray
this forum is our last resort plz someone help


----------



## 82ghost82

Hi my friends!!
I have a problem: I build a second workstation with 2x e5-2687w c2 with a second hand asus z9ped8 ws.
After I connected all the hardware when the workstation was powered on I got q-code b2 (I have a quadro 600).
I couldn't get into the bios setup.

After some trials I changed the bios chip from the working worstation to this second one and bingo! I can start the system.
This bios has v. 5404 (latest), whilst I don't know bios version of this second hand mb.

Now the question is: how can I update the bios if I cannot get into bios?is the only way to buy a bios chip with latest bios firmware?

Thank you


----------



## happy hopping

how exactly did you get the BIOS chip out in the 1st place? Shouldn't it be soldered?


----------



## 82ghost82

No, it's not soldered; all I had to do was slowly use a screwdriver and unplug it from the board


----------



## happy hopping

in that case, pull the BIOS chip that you want to upgrade to another Z9PE-d8 and upgraded it

Or that BIOS chip itself can be defective, in which you should just claim it under warranty


----------



## TheBlademaster01

If all you got was a b2 Q-code then it means the BIOS is incompatible with the VGA. Most likely trying an older GeForce GTX (500 and earlier) will POST on the BIOS.

Otherwise, you will need a new BIOS chip. Don't RMA the board, chances are you will get a defective one back from Asus and needless money spending on shipping (when a preflashed chip is mostly cheaper).


----------



## 82ghost82

Thank you Theblademaster01.
Yes, I was thinking it was about vga-bios incompatibility, but unluckly I haven't other vga to pull in the board.
The motherboard is not defective, so no rma.
I cannot update the bios in the other board because hardware is the same and I will get q-code b2 also on the other mb (I have not tryed but I think so..).

Going to buy on ebay a chip for 15 Euro..


----------



## TheBlademaster01

Before you order a new chip, there should be onboard VGA.


----------



## TheBlademaster01

Ah, but you need an adapter that is not included for that IIRC. Sorry, I think you need a pre flashed chip or different GPU


----------



## 82ghost82

Yes the blademaster, I think so...


----------



## Krobar

Other might disagree but I have hot swapped bios chips for this sort of thing in the past. Ie. boot of thee newer chip into dos, remove the newer version chip, insert the older version and run the Asus flash update.


----------



## 82ghost82

Quote:


> Originally Posted by *Krobar*
> 
> Other might disagree but I have hot swapped bios chips for this sort of thing in the past. Ie. boot of thee newer chip into dos, remove the newer version chip, insert the older version and run the Asus flash update.


Thanks, I thought also about it...did you have any damage?


----------



## Krobar

Have done it with Supermicros and Tyans a few years back and no problems although I suspect there is some risk.


----------



## 82ghost82

I have tried the bios update "on the fly" but it doesn't work








I tried both bupdater from dos and ezflah.
Ezflash says bios is outdated (???), bupdater gets stuked on loading screen...no luck today...


----------



## TheBlademaster01

You need to update to 3109 first before you flash to .CAP if you are still on .ROM bioses (pre 3109)


----------



## 82ghost82

Yes theblademaster I know, but error is the same..








Tried also afudos with no luck

Reading on this same topic it appears that there's a "me spi protection" on the board, so hot flash will not work..


----------



## 82ghost82

I'm having a new idea..I can remove the quadro 600 and enable the jumper to use onboard vga (default); I have not the idc16 to vga backplate, but I'm quite sure that if it boots I can remotely access the workstation without an additional graphic card.

To connect via lan for remote access I could start the workstation with the working bios chip, setup the lan and rechange the working chip with the older one.

The workstation should boot without b2 q-code, so it should enter windows and from a second pc I should access the workstation via remote access.

But the question is: what program (if it exists) can I use to update the bios from windows?I can't use "pure dos" since I have not direct vga output..I need something that can be done from windows os.

What do you think?


----------



## 82ghost82

Hi!
Again here for another issue









B2 bios error code was solved by installing an old asus graphic card and updating the bios to 3109, then to 5404.

And to reply one of my previous questions, yes, it is possible to start the system without additional graphic cards, nor flat cable to connect to vga port; of course the vga jumper on the motherboard must be in 1-2 position.
This can be very usefull for remote management.

Now...I saw 2 "strange" e5-2687w..in the metal part of the cpu there is the code sr0kg, which I thought it's c2 step..in windows control panel all the cores are indicated under "processors" as unknown device and in bios there is no description of the cpus. Low and high frequency are correct.

Cpuz indicates them as c0 step.

I saw different e5-2687w es and I also have to c0 step, but in the metal part there is the code qb7r, which is correct..

I-m attaching 2 pictures.





So, how to let control panel in windows recognize them with correct description?

Have you ever seen these type of cpus?

Thank you


----------



## TheBlademaster01

I believe there exist C0/C1/C2 E5-2687W. They are all pretty much retail, but C0 can also be ES.

Can you install HWINFO64?


----------



## GurKspaD

hallo i have a litle question do the Q19D 8c 1,6ghz work with the z9pe-d8 ? sorry fore my eng im from sweden :/

one more question i have a msi 660ti pe that whont boot whith the 5404 bios but it workt fine white 5206 do any one have a fix ore how to down grade the bios ?


----------



## 82ghost82

Quote:


> Originally Posted by *TheBlademaster01*
> 
> I believe there exist C0/C1/C2 E5-2687W. They are all pretty much retail, but C0 can also be ES.
> 
> Can you install HWINFO64?


Oh, yes, hwinfo shows my cpus as qb7r QS.
Strange thing is the sr0kg mark on cpus, as you can see in the second picture







.


----------



## TheBlademaster01

I have seen it more often tbh. In fact, a lot of Ivy chips are like that.

@Gurkspad

The chips don't post in Z9 D16, I'm not sure if it works with this board. I wouldn't recommend it either since it is only A1 stepping.


----------



## GurKspaD

any tips how i can get my 660ti working again then ?


----------



## GurKspaD

just geting past asus logo and then i just blinking line and b2 (legacy option ROM initialization)


----------



## 82ghost82

Quote:


> Originally Posted by *GurKspaD*
> 
> any tips how i can get my 660ti working again then ?


You can buy on ebay a chip, and request to the seller to flash v. 5206; you should spend about 10-15 Euro (about 20 $).

I'm not sure if and how you can downgrade the bios without a programmer.

PS: b2 in your case means you have rpoblem with graphic card; if you read the posts above I had similar problems, but in my case it was a problem related to older bios, which was solved by updating it. In your case you have to downgrade.


----------



## GurKspaD

Quote:


> Originally Posted by *82ghost82*
> 
> You can buy on ebay a chip, and request to the seller to flash v. 5206; you should spend about 10-15 Euro (about 20 $).
> 
> I'm not sure if and how you can downgrade the bios without a programmer.
> 
> PS: b2 in your case means you have rpoblem with graphic card; if you read the posts above I had similar problems, but in my case it was a problem related to older bios, which was solved by updating it. In your case you have to downgrade.


Quote:


> Originally Posted by *82ghost82*
> 
> You can buy on ebay a chip, and request to the seller to flash v. 5206; you should spend about 10-15 Euro (about 20 $).
> 
> I'm not sure if and how you can downgrade the bios without a programmer.
> 
> PS: b2 in your case means you have rpoblem with graphic card; if you read the posts above I had similar problems, but in my case it was a problem related to older bios, which was solved by updating it. In your case you have to downgrade.


ya but i dot have the time to wait im going to Dreamhack summer now the 14jun so it should work i i by a card from amd or the 700-seris ?


----------



## TheBlademaster01

You can downgrade without buying a new bios chip. You should try to post with nvidia 500 or amd 6000 and lower. What CPUs do you use?


----------



## GurKspaD

Quote:


> Originally Posted by *TheBlademaster01*
> 
> You can downgrade without buying a new bios chip. You should try to post with nvidia 500 or amd 6000 and lower. What CPUs do you use?


i have tried with my 470 and it starts but i got blackscreen in windows (probebly a drivers proble) i have a e5-2665 qa8z i only have one because i don't have so much money im ony 17 :/


----------



## TheBlademaster01

I have dual QA8Z and 780Ti. Best bios for me is 3302. I haven't tried 5206 and above. Never flash to 5004 btw.

Can't you just download a 3302 or 5206 if it worked from another PC and then flash the BIOS? You don't need to boot into windows.


----------



## GurKspaD

Quote:


> Originally Posted by *TheBlademaster01*
> 
> I have dual QA8Z and 780Ti. Best bios for me is 3302. I haven't tried 5206 and above. Never flash to 5004 btw.
> 
> Can't you just download a 3302 or 5206 if it worked from another PC and then flash the BIOS? You don't need to boot into windows.


i have aa 9800gtx that works i can go in to windows and everything but when i have the bios on a usb it says that i cant down flach the bios :/


----------



## TheBlademaster01

That's weird, I can flash from 3206 to 3506 just fine and vice versa


----------



## GurKspaD

Quote:


> Originally Posted by *TheBlademaster01*
> 
> That's weird, I can flash from 3206 to 3506 just fine and vice versa


i want to flash from 5404 to 5206 :/


----------



## TheBlademaster01

I know, but I haven't tried 5xxx because B0 doesn't work well with later revisions. I only have issues flashing back to 3109 and .ROM from 3506.


----------



## GurKspaD

Quote:


> Originally Posted by *TheBlademaster01*
> 
> I know, but I haven't tried 5xxx because B0 doesn't work well with later revisions. I only have issues flashing back to 3109 and .ROM from 3506.


how did you down flash ? o.0


----------



## TheBlademaster01

I go to ez flash in the bios and just select the older BIOS just as you would flash to newer BIOSes. When I select 3109 it says it's outdated or something and 0703 and before are incompatible file format.

3506 disables quad channel memory so I flashed back to 3302. 3206 doesn't post as consistently. 5004 doesn't post with ivy and non ecc and sandy B0. Other than that I don't know for sure,


----------



## GurKspaD




----------



## TheBlademaster01

Can you show which file you are flashing to?

I have no issues flashing to this

Z9PE-D8-WS-ASUS-3302.zip 3,834k .zip file


Z9PE-D8-WS-ASUS-5206.zip 4,146k .zip file


Make sure your USB stick is formatted as FAT32

Also, aren't you already running 5206 now? I see it noted in that screenshot.


----------



## GurKspaD

Quote:


> Originally Posted by *TheBlademaster01*
> 
> I have dual QA8Z and 780Ti. Best bios for me is 3302. I haven't tried 5206 and above. Never flash to 5004 btw.
> 
> Can't you just download a 3302 or 5206 if it worked from another PC and then flash the BIOS? You don't need to boot into windows.


I want to go to 3206  i have flasht down to 5004 now and now i cant go lower it says image outofdate(no it whont post whit the 660ti on 5004 bios to)


----------



## squidman

On ES E5-2665 b0 QA8X and bios 3506 (think it is a .cap bios) still working here. But BADLY want to upgrade via taobao to E5-2695 or E5-2697 v2's ....anyone got those C0 or similar working with non-ecc ram? And suggestions as to processor for this mobo? Need the highest median Mhz of course, not much point if all drops to low Mhz in the stepping once all cores are going.

And rumors of unlocked xeons? ES version Bx?

Back again, but goin trout fishing soon!

cookie:thumb:


----------



## GurKspaD

Quote:


> Originally Posted by *squidman*
> 
> On ES E5-2665 b0 QA8X and bios 3506 (think it is a .cap bios) still working here. But BADLY want to upgrade via taobao to E5-2695 or E5-2697 v2's ....anyone got those C0 or similar working with non-ecc ram? And suggestions as to processor for this mobo? Need the highest median Mhz of course, not much point if all drops to low Mhz in the stepping once all cores are going.
> 
> And rumors of unlocked xeons? ES version Bx?
> 
> Back again, but goin trout fishing soon!
> 
> cookie:thumb:


ya ty they say that q19d is unluckt but it is a a1 stepping but i dont rely know


----------



## TheBlademaster01

Quote:


> Originally Posted by *squidman*
> 
> On ES E5-2665 b0 QA8X and bios 3506 (think it is a .cap bios) still working here. But BADLY want to upgrade via taobao to E5-2695 or E5-2697 v2's ....anyone got those C0 or similar working with non-ecc ram? And suggestions as to processor for this mobo? Need the highest median Mhz of course, not much point if all drops to low Mhz in the stepping once all cores are going.
> 
> And rumors of unlocked xeons? ES version Bx?
> 
> Back again, but goin trout fishing soon!
> 
> cookie:thumb:


I heard rumours about unlocked HSW but other than that no.

No experience yet with those Ivy chips but they should work without ECC on 5206 and higher.

This seems like the best deal atm

http://item.taobao.com/item.htm?spm=a230r.1.14.24.yMWG0k&id=38443439159&ns=1#detail

Quote:


> Originally Posted by *GurKspaD*
> 
> ya ty they say that q19d is unluckt but it is a a1 stepping but i dont rely know


Q19D lacks turbo completely AFAIK


----------



## squidman

pretty sure I saw some 2.5 e5 2697's or 2695's at the same price (around 5 to 6k yuan)....are you using the same taobao as me? forget it..they changed the colors and front-end...


----------



## ZDngrfld

I'm running QDNR E5-2670 V2s without ECC. Have been for the better part of a year now.


----------



## 82ghost82

Hi!

I'm trying to force ram speed at 1600 Mhz and I have a problem.
I have 2 cpus and 64 gb rec ecc ram (Samsung id: M393B1K70CH0-CH9): these are 8x8gb DDR3 at 1333 Mhz

I went to bios and I only changed Advanced->Chipset configuration->Memory configuration->DDR speed and set it to force DDR3 1600.
When the system is restarted, if I go to bios one of the 8 slots in CPU1 is reading "Present 0 mb Independ", and in windows the system is using 58 gb of the ram (7 modules instead of 8).
However if I start HWinfo or CPUz they're reporting that my memory works in quad channel (???).

My tests:
- when forcing DDR3 speed to auto or 1333 Mhz bios and windows detect the correct amount of ram (about 65 gb), so the ram is ok and the motherboard is not defected
- if I swap ram modules and I force speed to 1600 Mhz the slot reporting 0 mb is always the same.

I think my problem is the voltage, but I cannot find how to increase dram bus voltage, I can only change QPI/DRAM voltage.

So, anyone know how to increase the bus voltage?It seems it cannot be set....

Thanks to anyone who can help.

Daniele


----------



## TheBlademaster01

I must say that I have the same problem, although I think it sees all RAM in Linux. Have to check, in windows it also reported 28 GB RAM and Quad Channel. I haven't really looked into it.

As for the memory controllers, they are integrated on the un-core (System Agent). the voltage should be named something like CPU1_SA or so. Not sure if it will solve the problem though. I noticed that BIOS 3506 saw all of my RAM at 1600MHz but it did so at triple channel.










E:

Seems Windows also saw 32GB. Strange...


----------



## 82ghost82

Hi theblademaster, thanks for your replies.
Hwinfo and cpuz see 64 gb ram and quad channel, but control panel and bios see 65-8 gb.
If I run prime95 setting the ram to be used at a value of 64 gb the system only use 65-8gb...

So are you saying that to change dram bus voltage I should change cpu vsa voltage?

Thanks


----------



## TheBlademaster01

Yes, you're correct. It still sees 32-4GB of RAM



I can't really have downtime atm otherwise I would have looked into it. But yes VSA should be the voltage for the memory controller. I recall that anything over 1V is risky and will rapidly lead to migration/tunneling = kaput.


----------



## 82ghost82

Ok, problem solved, thank you very much theblademaster, you are my personal guru









I had to increase a little bit dram and vsa voltages and relaxing primary timings; now bios detects correctly 65 gb.

Will it be safe with that settings?


----------



## TheBlademaster01

Haha, no problem. SA voltage seems okay 0.98V is pretty decent. I had my 2600K at 0.985V most of the time (3 years).


----------



## vgame64

Hi folks, I'm in need of some help.
Q1: Has anyone had a good experience with 3 way, or 4 way SLi/CrossFireX?

(I've got 2 Ivy Bridge Xeons installed)

I attempted to run 4 R9 290s, but my experience in benchmarks with Crossfire profiles was extremely choppy.
I then switched to 3 R9 290s (in the top 3 blue PCIe slots), and the experience was improved, but still not optimal.

Next, I moved to 2 R9 290s in the top blue slot, and the third blue slot, and I tried Bioshock Infinite...left to right movements were extremely jerky and filled with tearing.

I then tried 2 R9 290s in the top blue slot, and the second blue slot, and Bioshock Infinite was perfectly smooth.

Q2: Is the Z9PE-D8 unsuitable for 3 or 4 way GPU setups for gaming due to the way the PCIe lanes are split between the CPUs, or is this more pronounced on the R9 290s due to their utilization of a DMA engine to communicate with each other instead of a Crossfire/SLi bridge?

Thanks!


----------



## vgame64

To answer my own question(s) and help anyone in the same unique situation I am in, yes, the Z9PE-D8 is unsuitable for 3 or 4 way CrossfireX with the R9 290/X graphics cards.

I tested with 2,3, and 4 GTX Titans, and all of my issues disappeared.


----------



## TheBlademaster01

It could very well be the DMA and the fact that lanes 1 and 2 (blue ones) are ran off CPU0 and 3 and 4 are ran of CPU1. Though I've never looked into DMA much tbh









E:

Just looked into it. Yes, the culprit really is the DMA engine. Since on multi node systems the "host" needs to communicate between 2 controllers. It would add severe latency. This design is good for parallel computing, but not so much for gaming. A better option in this case would be the SR-X since I believe that all PCI-E lanes are ran of one CPU while the other runs SATA and USB buses IIRC. You'd have to make sure about the SR-X if you're really interested though. It would be worse for parallel computations however.


----------



## vgame64

Thanks Blademaster.


----------



## tgurus

Hey, I am having an almost similar DIMM problem -

My DIMM slot B shows as "Present" but "Mirrored" in the BIOS (with 0GB avail)

Only 56GB instead of 64GB showing avail in BIOS and Windows both (whether I have a stick in the slot or not)

Have tried multiple different configs and DIMM sticks (even different brands)

Rt now have a low profile Crucial Ballistix (1.35V) with Auto voltage showing 1.34V

Processor is the E2697v2 and VCORE auto voltage is showing .946V and VSA .998V for the pertinent CPU

Any ideas?


----------



## Stefan7

Midway through my build waiting on the last couple of bits and bats to arrive. And waying up the last components

Took a gamble on some second hand parts :/ because i could not afford to build this with 100% new components!

Building this as rendering station for revit speciffically, as revit doesnt currently support GPU rendering which is absolutely absurd.

*Specs and prices of what ive purcahsed so for in £ As follows including shipping*.

Asus Z9PE-D8 Mobo (obviously) £270 Second hand
silverstone GD07 Case £95 New (Tiny bit chopped out of drive bay holder for one fan)
2x Xeon E5-2665 CPU's £400 (For two) Second hand
2x Noctua Nh-L12 Coolers £80 (For two) New
Nvidia Quadro K4000 3gb GPU £380 Second Hand
32gb Corsair xms3 1600mhz Ram £200 New 4 x 8gb (really would like to up this to 64gb)
Crucial M500 480gb SSD (Start up) £150 New
Corsair HX850 PSU £66 Second Hand
LG BH16NS40 Blu-Ray rewritter £62 New with software disc (alot of them dont have disc)
Akasa Card reader with Bluetooth £16 New
Gelid Speed touch 6 fan controller £40 New
2 x Be quiet Silent Wings 2 120mm £37 (For two) New
2 x Be quiet Silent Wings 2 80mm £22 (For two) New
£1818

*Other Stuff Bought*
Mele F10 Pro remote £26 New
Underdesk Desktop Mouting Brackets £20 New
£1864

*Still weighing up / still need*

Storage Hard Drive 2TB? is one just as good as another there around the £50 mark

Sound Card....needs to have a decent line in as i do some music recording. What to buy could do with a phono input? £50 max

Wifi Network Card or Dongle...?...can anyone spread some light on this which is better my router is 2 floors away. Should i get one with an Antenna? £25 max

I might swop the Noctua fans in the coolers for Silent Wing 2's so that would be another 2 x 120mm and 2 x 92mm fans £60 max

Wireless Illuminate keyboard £50?
£2100

*In the pipe line.*

Fully articulated gas sprung dual monitor stand. £105

Two more extra monitors £200 max

Extra 32gb Ram £250 Cant get it for £200 again
£2650

Close wallet, Ration food for rest of year


----------



## 82ghost82

Quote:


> Originally Posted by *tgurus*
> 
> Hey, I am having an almost similar DIMM problem -
> 
> My DIMM slot B shows as "Present" but "Mirrored" in the BIOS (with 0GB avail)
> 
> Only 56GB instead of 64GB showing avail in BIOS and Windows both (whether I have a stick in the slot or not)
> 
> Have tried multiple different configs and DIMM sticks (even different brands)
> 
> Rt now have a low profile Crucial Ballistix (1.35V) with Auto voltage showing 1.34V
> 
> Processor is the E2697v2 and VCORE auto voltage is showing .946V and VSA .998V for the pertinent CPU
> 
> Any ideas?


Is "memory mode" in advanced tab set to independent?
Are hwinfo/cpuz showing quad mode?


----------



## tgurus

Thanks for the reply--
Quote:


> Originally Posted by *82ghost82*
> 
> Is "memory mode" in advanced tab set to independent?


YES. This is also the only option available. Other than the bad slot, they all show as Independent when you drill down
Quote:


> Originally Posted by *82ghost82*
> 
> Are hwinfo/cpuz showing quad mode?


Well, these are 12-core-- all 12 show up for each CPU in Resource Monitor as unparked and System Info shows each as 12-cores
(hwinfo gives me a nice BSOD)


----------



## TheBlademaster01

No, he means quad channel memory 

Is it always the same slot that malfunctions no matter how many slots you populate?


----------



## tgurus

Quote:


> Originally Posted by *TheBlademaster01*
> 
> It could very well be the DMA and the fact that lanes 1 and 2 (blue ones) are ran off CPU0 and 3 and 4 are ran of CPU1.


Interesting. So if I have 2 GPUS, and 2 CPUS, I really want to have the GPUS in slots 1 and 3, eh?


----------



## tgurus

Quote:


> Originally Posted by *TheBlademaster01*
> 
> No, he means quad channel memory
> 
> 
> 
> 
> 
> 
> 
> 
> 
> Is it always the same slot that malfunctions no matter how many slots you populate?


Yes. It's the same slot every time... beginning to wonder if it's just a bad slot physically...


----------



## TheBlademaster01

Quote:


> Originally Posted by *tgurus*
> 
> Interesting. So if I have 2 GPUS, and 2 CPUS, I really want to have the GPUS in slots 1 and 3, eh?


If you're gaming, yes. For parallel computing it's best to spread the GPUs over the CPUs (low latency between host memory and GPU)

Quote:


> Originally Posted by *tgurus*
> 
> Yes. It's the same slot every time... beginning to wonder if it's just a bad slot physically...


99% sure it's a faulty slot (which is a pretty common reason for DOA on this board). The Z9PE-D8 WS has a very high failure rate sadly


----------



## 82ghost82

Run cpuz and see the memory tab if it reads 64 gb and 8 gb per slot; run hwinfo, but before running in preferences check to not scan for ide/scsi drives (this should solve the bsod) and see also if hwinfo sees 56-57 or 64 gb.


----------



## AceCobra1

Quote:


> Originally Posted by *TheBlademaster01*
> 
> I recognize your username MunneY, but I don't know from where.
> 
> Anyways. Sandy Bridge B0 don't work properly with .CAP bioses (every BIOS release after 0703). Ivy Bridge only works from BIOS version 5004 and later, but 5004 only works with ECC RAM if you have an Ivy Chip. BIOS 5004 also completely breaks compatibility with Sandy B0, so they won't post anymore. QS Sandy Bridge C0 and later work with all BIOS revisions.
> 
> You also need BIOS 3506 and later to have the board recognize PCI-E 3.0 cards properly (Kepler, Tahiti, Maxwell etc.)
> 
> If you use PCI-E 2.0 cards only you could get away with B0 chips and absolutely not flashing to BIOS 3109 and later. After that it is hell (my situation
> 
> 
> 
> 
> 
> 
> 
> ). It also won't work with Windows 8 on those older BIOSes.


I just bought a QA82 ES and my z9ped8 turns on but is refusing to boot - do you know exactly which bios I need to use? I have bought one off ebay to replace my existing bios (not sure what version it is as I bought the board 2nd hand). I am planning to buy a 2nd QA82 ES as well if it runs the first one. Also, with regards to RAM, do you know if there are specific ones I need? Some of them say you need ECC and non ECC ram - not sure what they are.

I just had a look at this:

http://www.ebay.co.uk/itm/Intel-Xeon-E5-2650-ES-QA82-LGA2011-2-0Ghz-2-3Ghz-Turbo-8C-16HT-20M-L3-B0-Step-/331180359028?ViewItem=&item=331180359028&nma=true&si=L7TUDwpzb7RcvYR96OloYx2vjfc%253D&orig_cvip=true&rt=nc&_trksid=p2047675.l2557

Does that mean I should buy a BIOS flashed with ver 3103 ?


----------



## TheBlademaster01

They "post" on anything not 5004 as far as I know. They only post consistently on BIOSes lower than 3109. Though you need 3302 or later if you have PCI-E 3.0 graphics cards. RAM is irrelevant mostly.


----------



## AceCobra1

Thanks for your reply. When you say they post _consistently_, does that mean that bios 5004 may or may not post sometimes ? I will mainly be using the motherboard + CPUs to mine cryptocurrencies so I don't think graphics card are relevant in this case. So basically, I can use any old graphics card to run the board but just at slower speed right ? What bios would you recommend me to purchase ?


----------



## TheBlademaster01

I'd recommend 0503 in that case. And no, 5004 will not post at all. 3109 and later (except 5004) will post once in a couple of cold reboots (ranging from 3-7 cold boots from my experience). 0503 posts as a normal system would


----------



## AceCobra1

Quote:


> Originally Posted by *TheBlademaster01*
> 
> I'd recommend 0503 in that case. And no, 5004 will not post at all. 3109 and later (except 5004) will post once in a couple of cold reboots (ranging from 3-7 cold boots from my experience). 0503 posts as a normal system would


Thanks buddy - so in terms of GPUs, would it work with anything? I mean if i plugged in my 750Ti, it will still display but if I were to play games, it will bottleneck as it will slow down right? For mining purposes, it doesn't matter weather the GPU is plugged into a 1x or 16x PCI-E, they still perform equally well


----------



## TheBlademaster01

No problem









No, PCI-E 3.0 video cards will make it fail the POST process (Q-code b2) or simply not detect the GPU and have no display if BIOS version is older than 3302.

Only GTX 500 and older and HD6k and older will post. I believe some Quadros and FirePros won't POST either.


----------



## AceCobra1

Ah - I guess thats the only problem running these cheap ES from intel then ?

I hope my GTX 570 works... I have a HD4870 as well







old school ones.

Why don't Asus make a bios that will work with both the newer PCIE 3.0 that also works with the ES CPUs as well ?


----------



## 82ghost82

Quote:


> Originally Posted by *AceCobra1*
> 
> Ah - I guess thats the only problem running these cheap ES from intel then ?
> 
> I hope my GTX 570 works... I have a HD4870 as well
> 
> 
> 
> 
> 
> 
> 
> old school ones.
> 
> Why don't Asus make a bios that will work with both the newer PCIE 3.0 that also works with the ES CPUs as well ?


Because we shouldn't have ES cpus







, so no need to have updated bios for ES.


----------



## TheBlademaster01

Quote:


> Originally Posted by *82ghost82*
> 
> Because we shouldn't have ES cpus
> 
> 
> 
> 
> 
> 
> 
> , so no need to have updated bios for ES.


Pretty much this. I do think Supermicro boards support B0 Sandy and PCI-E 3.0 properly though. In fact, I heard rumors about the Z9PE-D16 even properly supporting them


----------



## ironcurtain

I'm getting 6C code with no post in a new build.

2 x 2630 v2
32G (4 x 8G 16G kit Crucial Ballistix tactical PC3-12800 non ECC)
PSU is Corsair AX1200

All disks disconnected, I removed the GPU too (GTX 570 HD EVGA), didn't help...

At some point it did start when I was testing the build to a 00 code... now it gets stuck to 6C.

ECC memory shouldn't be required and I got none to test


----------



## ZDngrfld

Quote:


> Originally Posted by *ironcurtain*
> 
> I'm getting 6C code with no post in a new build.
> 
> 2 x 2630 v2
> 32G (4 x 8G 16G kit Crucial Ballistix tactical PC3-12800 non ECC)
> PSU is Corsair AX1200
> 
> All disks disconnected, I removed the GPU too (GTX 570 HD EVGA), didn't help...
> 
> At some point it did start when I was testing the build to a 00 code... now it gets stuck to 6C.
> 
> ECC memory shouldn't be required and I got none to test


Do you know what version BIOS you're running? You need to be running at least 5103 to support IVB-E, but you need to be running 5206 to support IVB-E WITHOUT having to run ECC. If you're running an older version you'll need to throw a SB chip in to flash or you can buy a new BIOS chip with the latest version off of eBay if need be.


----------



## ironcurtain

I took everything off and plugged an USB stick with an updated BIOS in.... it worked, I tested part by part and now it seems to be working. Unfortunately I had to fix a bent pin in the CPU2 socket because I got some cloth tangled while cleaning a thermal grease leak (from the Coolermaster 212... I put excess grease there and maybe it was capacitive/conductive, shorting the pins).I hope I can keep the board, altohugh you probably will recommend returning it (Amazon purchase..)

I had to force 1600 speed through BIOS, it was defaulting to 1333. Also it seems the QPI speed is 7.2 instead of the full speed. Do I need another 2x8GB kit (16GB) to populate one socket in each empty DIMM bank left?

Thanks!


----------



## GurKspaD

hallo i have some problem with my 470 freezing in windows i can start in safe mode and it works fine but after login screen it freezes after like 10sec


----------



## 82ghost82

This sounds like a driver issue.Make sure to install latest version.


----------



## ironcurtain

I'm experiencing some really horrible compat issues with Samsung 840 EVO SSDs... 1TB model. If I don't install the AHCI drivers, I can see good performance AND use the passthru hardware encryption for Bitlocker.... (after secure erase, fresh install, enable bitlocker). After installing the AHCI drivers bitlocker breaks and appears turned off while the SSD remains 'locked' (needs PSID revert). I tried to enable AFTER installing the drivers.... same problem. The performance is also terrible.

Any ideas? I think I will try to disable the LSI controller via jumper and use the Intel one directly. I have my two SSDs plugged to the blue sockets.


----------



## MaDDeePee

Anyone tried new Version 5503? (2014/06/11)

It says:

1. Support OC profile.
2. Improve the system stability.


----------



## GurKspaD

Quote:


> Originally Posted by *MaDDeePee*
> 
> Anyone tried new Version 5503? (2014/06/11)
> 
> It says:
> 
> 1. Support OC profile.
> 2. Improve the system stability.


ya i flasht today. I dint notest anyfing XD


----------



## Panther07

Hi All

New owner of this board, very happy with it - it's in another league to anything I've had before!! This thread is absolutely great and it helped me to piece my dream computer together







hence I've joined the forum.

Just a couple of questions if anyone can answer:

1) I'm running Windows 8.1 and I'm unable to properly install the Intel drivers for the two on-board 82574L NICs - I want to team these to a gigabit HP switch but I just can't get the teaming software installed properly (fails at the driver stage). The default driver Windows has put down runs the NICs fine and they show up in device manager no problem - any ideas?

2) I've read that the on-board LSI MegaRAID isn't too shabby, especially not with Windows 8.1 - can anyone give me their experience? I'm actually using the Intel Raid at the moment, should I just stick with that?

3) Anything else you can recommend as a starting point with this board?

Cheers


----------



## artzgo

Quote:


> Originally Posted by *MaDDeePee*
> 
> Anyone tried new Version 5503? (2014/06/11)
> 
> It says:
> 
> 1. Support OC profile.
> 2. Improve the system stability.


OC profile cool, but OC board poor


----------



## halkx

Question the psusmb1 connector on the upper right side of the motherboard is for connecting to a psu...what for? I am waiting for the ax1500i to replace my trusty old ax1200 and there is a similar connector and cable...can I plug it there?


----------



## ironcurtain

And we are back to 6C Q code and no post after BIOS upgrade.... what is happening now? Trying to reset BIOS doesnt seem to help.


----------



## ironcurtain

Quote:


> Originally Posted by *ZDngrfld*
> 
> Do you know what version BIOS you're running? You need to be running at least 5103 to support IVB-E, but you need to be running 5206 to support IVB-E WITHOUT having to run ECC. If you're running an older version you'll need to throw a SB chip in to flash or you can buy a new BIOS chip with the latest version off of eBay if need be.


I previously fixed this somehow, now it's back to non/operational....I simply rebooted after updaing the BIOS to the latest version as of today. It's once again not posting. I cleared CMOS and plugged a Kingston 16GB drive with the converted BIOS, and this didn't help. I haven't taken out the CPUs or DIMMs yet, because this would be the third time I go through all the mess.

What can I do?


----------



## AceCobra1

OK, I was a dumb dumb for not reading this thread earlier... Turns out my board wasn't booting because of me using a PCIE 3.0 GPU














I am running the QA82.... Thing is, I moved my harddisc from my desktop to the WS motherboard.... It is an SSD.... My god, it takes 5 mins to boot into windows 7?? Any ideas? Then when I get in, windows run so slowly... Even the mouse jerks around







Any ideas?

GPU 570 GTX
PSU 1200W Cooler Master Silent Pro Gold
12 GB RAM on slots A,C,D

Motherboard code shows 22..

Is it ment to light up with some sort of code all the time?? Does it ever go off?


----------



## ccRicers

Quote:


> Originally Posted by *TheBlademaster01*
> 
> No problem
> 
> 
> 
> 
> 
> 
> 
> 
> 
> No, PCI-E 3.0 video cards will make it fail the POST process (Q-code b2) or simply not detect the GPU and have no display if BIOS version is older than 3302.
> 
> Only GTX 500 and older and HD6k and older will post. I believe some Quadros and FirePros won't POST either.


GTX 580s in dual SLI would work I hope, right? Wanting to SLI to have performance catch up more with modern cards.

Also I'm going to be using a ES pair of the E5-4610, C1 stepping though... not listed on the spreadsheet here so I'll have to post back for results.


----------



## AceCobra1

Quote:


> Originally Posted by *ccRicers*
> 
> GTX 580s in dual SLI would work I hope, right? Wanting to SLI to have performance catch up more with modern cards.
> 
> Also I'm going to be using a ES pair of the E5-4610, C1 stepping though... not listed on the spreadsheet here so I'll have to post back for results.


I have a problem. The QA82 E5-2650 which I am using is actually OK - turns out my cooling wasn't working properly... Anyway, I installed my old GTX 570 Asus into the motherboard. Boots up fine and all but when I boot into windows and install the nvidia drivers (tried several), it causes it to freeze after around 10 seconds of being in windows. I suspect this maybe a driver issue.... My graphics card never had any issues since I removed it from my old system 2 years ago. I tried it on the Asrock X79 Extreme 9 motherboard and that also caused it to freeze. Not exactly sure what is wrong. Even tried reinstalling windows etc. Using different harddiscs and all... still same issue. Now I am resorting to playing Dota on my HD4870... My god... so slow


----------



## AceCobra1

Do you guys know if the GT730 will work with the QA82 CPU as it seems that it is a PCIE2.0 ?


----------



## ccRicers

Quote:


> Originally Posted by *AceCobra1*
> 
> I have a problem. The QA82 E5-2650 which I am using is actually OK - turns out my cooling wasn't working properly... Anyway, I installed my old GTX 570 Asus into the motherboard. Boots up fine and all but when I boot into windows and install the nvidia drivers (tried several), it causes it to freeze after around 10 seconds of being in windows. I suspect this maybe a driver issue.... My graphics card never had any issues since I removed it from my old system 2 years ago. I tried it on the Asrock X79 Extreme 9 motherboard and that also caused it to freeze. Not exactly sure what is wrong. Even tried reinstalling windows etc. Using different harddiscs and all... still same issue. Now I am resorting to playing Dota on my HD4870... My god... so slow


I actually haven't gotten a chance to use this motherboard yet. Maybe I'll test boot it today just to see if it POSTs and gets to the Bios. Would be able to try with a single chip although I still need a compatible GPU to test with.


----------



## TheBlademaster01

Quote:


> Originally Posted by *halkx*
> 
> Question the psusmb1 connector on the upper right side of the motherboard is for connecting to a psu...what for? I am waiting for the ax1500i to replace my trusty old ax1200 and there is a similar connector and cable...can I plug it there?


It seems to enable some sort of controller on the motherboard to connect temperature and power sensors with. Not certain of the applications for it however. I didn't find much about it on the internet.

http://www.manualslib.com/manual/10138/Asus-Motherboard-Psch-Sr.html?page=41

Quote:


> Originally Posted by *AceCobra1*
> 
> OK, I was a dumb dumb for not reading this thread earlier... Turns out my board wasn't booting because of me using a PCIE 3.0 GPU
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> I am running the QA82.... Thing is, I moved my harddisc from my desktop to the WS motherboard.... It is an SSD.... My god, it takes 5 mins to boot into windows 7?? Any ideas? Then when I get in, windows run so slowly... Even the mouse jerks around
> 
> 
> 
> 
> 
> 
> 
> Any ideas?
> 
> GPU 570 GTX
> PSU 1200W Cooler Master Silent Pro Gold
> 12 GB RAM on slots A,C,D
> 
> Motherboard code shows 22..
> 
> Is it ment to light up with some sort of code all the time?? Does it ever go off?


It changes depending on the OS. My motherboard used to display Q-code 15 in Windows, but after flashing to .CAP it shows 22. Both are fine as far as I know. And yes, they are supposed to stay on. Only during shut down or system error (BSOD/kernel panic) the codes change.

In Ubuntu I used to get AA, in openSUSE I properly get 00 and in elementaryOS I get 22 as well.

I'm not sure if C0/C1 or proper retail CPUs also show 22 in OS though.

Quote:


> Originally Posted by *ccRicers*
> 
> GTX 580s in dual SLI would work I hope, right? Wanting to SLI to have performance catch up more with modern cards.
> 
> Also I'm going to be using a ES pair of the E5-4610, C1 stepping though... not listed on the spreadsheet here so I'll have to post back for results.


C1 should work with any conventional GPU setup. It is Sandy B0 + PCI-E 3.0 that gives issues. Do note that PCI-E 3.0 cards need 3302 and later to POST with Sandy C0 and later.

Quote:


> Originally Posted by *AceCobra1*
> 
> I have a problem. The QA82 E5-2650 which I am using is actually OK - turns out my cooling wasn't working properly... Anyway, I installed my old GTX 570 Asus into the motherboard. Boots up fine and all but when I boot into windows and install the nvidia drivers (tried several), it causes it to freeze after around 10 seconds of being in windows. I suspect this maybe a driver issue.... My graphics card never had any issues since I removed it from my old system 2 years ago. I tried it on the Asrock X79 Extreme 9 motherboard and that also caused it to freeze. Not exactly sure what is wrong. Even tried reinstalling windows etc. Using different harddiscs and all... still same issue. Now I am resorting to playing Dota on my HD4870... My god... so slow


Faulty card, typically the memory from my experience with an old GTX 285 and 9800GTX. You might want to try a custom VBIOS with more supplied voltage to the chip in case RMA is not an option. Otherwise I'd urge you to RMA.

The thing is that after installing the drivers, the GPU is ramped up to 3D clocks from its idle 2D state (for Fermi this is 405MHz core, 810 MHz shader and 300'ish MHz IMC from the top of my head). Most likely the GPU is still stable in its 2D state but becomes unstable when at full clocks.

Quote:


> Originally Posted by *AceCobra1*
> 
> Do you guys know if the GT730 will work with the QA82 CPU as it seems that it is a PCIE2.0 ?


Not a lot of chance IMO. Though no one knows for sure without trying 

I highly doubt it however. Quadro K5000 is a PCI-E 2.0 card as well and doesn't "work" either. That said, I'm currently typing this from my 2x QA8Z + 780Ti system that posts like 1 in 3-5 times with this GPU on v3302 

Best option for QAxx chips is to really stay on 0703 and older with GTX 500 and older or HD 6000 and older. Anything other than that will give you headaches.

Quote:


> Originally Posted by *ccRicers*
> 
> I actually haven't gotten a chance to use this motherboard yet. Maybe I'll test boot it today just to see if it POSTs and gets to the Bios. Would be able to try with a single chip although I still need a compatible GPU to test with.


Should work. Notice that the first socket is the rightmost socket and the DIMMs surrounding the other socket + the lower 3 PCI-E slots won't be operational in single CPU mode.


----------



## TheBlademaster01

Quote:


> Originally Posted by *ironcurtain*
> 
> I previously fixed this somehow, now it's back to non/operational....I simply rebooted after updaing the BIOS to the latest version as of today. It's once again not posting. I cleared CMOS and plugged a Kingston 16GB drive with the converted BIOS, and this didn't help. I haven't taken out the CPUs or DIMMs yet, because this would be the third time I go through all the mess.
> 
> What can I do?


Swapping the CPUs sometimes forces the IMEI to reset, which may help it to POST.

What exactly is the BIOS version you're currently running and SPEC of the CPUs? (retail?)


----------



## AceCobra1

Any suggestions as to which bios i should flash to on my GTX 570 ?


----------



## 82ghost82

I confirm qcode 22 with c1 cpus on windows.


----------



## Big Jim

I am getting a QCode OE (microcode not found) at start up, then the system hangs there and the bios never finishes loading. I cannot get back into system. Prior to this the system would lock up after windows 7 64-bit booted up for a few minutes. The manual offers nothing on how to fix this, just the code table. Was trying to upgrade BIOS to 5503. After rebooting system after running Flash Bios, computer locked down at "OE" QCode during reboot and trying to clear cmos failed as did removing the battery. System locks up before, I can relaunch usb drive to reload bios using the BUPDATER.Exe. Asus said to remove memory, videocards and cpu's to check, system wouldn't start without both cpu's and memory installed. Ended with same Q Code OE. What, can I do to fix this and make system stable and not freeze up after loading Windows 7 64-bit.

Running Intel Xeon 2620 V2, Kingston Value Ram 8-16mb sticks for a total of 128 mb ram. 2- Nvidia Geforce 670 4mb videocards w/SLI, Samsung 1Tb Solid State Drive, 4- Seagate 3Tb hard-drives In a Cooler Master Cosmos ll Case and powered by a Corsair AX1200i. Using 2- Intel Thermal Solution STS200C to cool processors.


----------



## TheBlademaster01

Sorry, I can't really help with that. I can only advise you to try booting with 1 CPU and 1 DIMM in it and a GPU. I had 0E before, but that was with my B0 chips in Windows. The system would lock up for a couple of minutes and then be responsive again (repeating the cycle).

Not sure how it relates to your Ivy chips. I have heard of 2620V2 and 5503 POST issues before. I believe it was ironcurtain above. Are these chip Engineering Samples?

@AceCobra

Like I said, 0703 or before. 0503 worked best, but you can't flash back that far. You'd have to buy a pre-flashed chip from eBay.

@82ghost

Thanks


----------



## Big Jim

The Intel 220 v2 Zeno were purchased through Amazon. Com. Looking at date of release I thought the 5503 was the latest release. Since computer kept locking up after 3-5 minutes, I thought upgrading the Bios would help. Now, I have QCode OE and everything stops. Cannot get back in to reload Bios. Manual only displays code and offers no solutions to the QCodes. The computer wouldn't even start with only one cpu,one videocard and one ram chip.


----------



## Big Jim

Intel xeon E5-2603v2 six - core 2.1GHz 7.2GT/s 15MB LG 2011 CPU Amazon @ $412.03 each. The Ram is Kingston Value Ram 64 gb (4x16Gb) 240 pin DR 1600 ETC Registered Server Memory (Intel Validated) Model KVR16R11D4K4/64 @ Egghead $596.99 for 64 gb.
So, my system has all the same ram. I filled all 8 slots to a total of 128Mb, system saw 98 mb and the rest in reserve.

Not sure why it would freeze after a few minutes windows 7 64-bit loaded. Or what the QCodes OE means other than microcode not found. No idea on how to clear and get system back up and running. Keyboard non-responsive once QCode OE appears.


----------



## Big Jim

I contacted Asus Support about the QCode OE error. Asus Support had no idea what it meant or how to fix problem. Offered to repair Motherboard due to warranty coverage. I set it out for repair. The options I was given was repair my Board or get a reconditioned replacement instead. The difference being 7 days return on my Board or 5 days on the reconditioned board. I decided, I knew the history of my board and a few days more wouldn't make a big enough difference in the repairing. I requested that Asus include the correct Bios settings for my components and included with the motherboard a list of my CPU's, Ram, Hard-drives, and Video cards. Hopefully Asus will respond with the correct settings to use and fix the QCode OE error. I did order a replacement Bios Chip from Asus to keep on hand as a back up, Which did arrive today. I couldn't see any cracks or other problems on the motherboard, prior to sending it in for repair. Still no idea on the "Microcode not found" problem or how to prevent that error in the future.
I did try to clear the cmos and took out all but one CPU, one Ram stick, one video card. System halted on Q Code 60 and failed to move forward, keyboard non-responsive and no way to load usb CrashFree Bios. At this point I removed all components and boxed motherboard for shipping for repair. I was surprised that Asus didn't tell me to go back to retailer for repair. Most other Companies these days tell you it's not their problem go back to retailer and Then the retailer tells us the same thing and we end up stuck in the middle, in no where land. So, it surprised and relived me that Asus didn't play that game. I hope they find the problem and fix it.


----------



## drjam

Hi All, i joined simply to ask something in this thread, as teh entire internets doesnt have this out there!
(also, much forgives pls, if there is another place to post this in here...)

I have installed an ASUS Server TS700-X7/PS4, which of course has the Z9PE-D8 MB in it. It has 4x 8GB ECC non-reg memory with the currently installed XEON.
I want to Max out the RAM without using the second CPU socket.
For search engines: I want to use 64GB of RAM but only ONE CPU socket in the Asus Z9PE-D8.

Can this be done? Have any of you lads performed such a feat?
This is a production server, so I cant simply test it out ya know.

Thanks in advance,


----------



## TheBlademaster01

Only registered RAM can do this. Each CPU is limited to 32GB non-registered RAM per CPU (in 4x 8GB config).

If you want to use higher density RAM you need registered memory, because the register between the RAM and the CPU adds more signal integrity. You can use up to 4x the capacity with registered RAM, but it is slower.

@Big Jim

Thanks for the info. Please report back if you get the board out of RMA. Will definitely be useful for other people.


----------



## MaDDeePee

Sooo...did i get that right, is it a bad idea to flash to BIOS 5503 from BIOS 5404 ?

dual 1,6Ghz QDU4 ES2 (ivy)
16Gb non-ecc
Geforce 660ti

right now with 5503 i seldom and randomly having problems while rebooting (not from cold start) with hanging while BIOS post with blackscreen (don't remember code yet but if i remember right it was PCI-Express related).
Need to hard-reset with power switch to make it boot again...


----------



## ironcurtain

Hey guys,

System running smooth but.... there's a very faint hissing intermittent beep going on. I don't think it belongs elsewhere but the case. Has anyone experienced this with the mobo?


----------



## ironcurtain

I can't install the Intel NIC drivers on Windows 8.1 64-bit and I need the damn teaming ANS software to configure link aggregation... has anyone had any success with this?

Edit: It seems Intel does not support Windows 8.1 x64 for the drivers... and Microsoft bundles their own with no link aggregation support. Isn't this great.


----------



## squidman

Hi Sorry new topic (link aggregation would be great for me too! but not much point as am only on 50 down/10up subscription...maybe iff I get around to building a Norco 4224 storage server)....

OK actually 2 questions (on z9pe-d8 ws with 2x QA8X e5-2665):

1) I read somewhere that it was possible to force the turbo to max on all cores, but haven't figured out the bios settings...did successfully dial the bclk up a tiny notch or two, but bit pointless so set it back to automatic....rendering at the moment my 2665's pretty much max out at 2.4ghz...which still sucks for e.g. Vue Xstream or stabilizing with Mercalli...SLOOOW....on the consumer boards the feature was called ASUS something or other....

2) Has anyone got the IRSTe software to work? Have 4 WD red 4tb in raid 5 on the bottom 4 SATA/SCU ports, no probs with the intel raid functionality in BIOS, but the IRSTe software always crashes. The marvell web-based gui works fine for me (ssd caching). Should i uninstall (hoping that won't mess with my raid...would be catastrophic) and start from scratch? Wondering whether I have to reset the OPROM settings in BIOS. In fact I'm a bit confused...is the bios IRSTe OPROM different and separate from the Windows IRSTe utility? I thought the latter was just a windows gui (like the marvell web-based utility) to monitor the raid set....

Thanks! Good to be back, will have more questions on Taobaoing some e5-2695/7's in a bit in a few posts (NEED cinebench 11.5 score of 34!)....

Shame that Algeria didn't win...they would have been facing france in the quarter finals the day after Algerian independence day, a chance to reallly stick it to them!


----------



## drjam

Quote:


> Originally Posted by *TheBlademaster01*
> 
> Only registered RAM can do this. Each CPU is limited to 32GB non-registered RAM per CPU (in 4x 8GB config).
> 
> If you want to use higher density RAM you need registered memory, because the register between the RAM and the CPU adds more signal integrity.


Thanks for that reply! It answers my questions.
Funny this isnt posted on the official site somewhere.
A digital high-five is coming your way man!


----------



## TheBlademaster01

Quote:


> Originally Posted by *ironcurtain*
> 
> Hey guys,
> 
> System running smooth but.... there's a very faint hissing intermittent beep going on. I don't think it belongs elsewhere but the case. Has anyone experienced this with the mobo?


Yes, it is coming from the power regulators I think. Had that since day one, and I think it is normal when dealing with this kind of power delivery. Not sure about the beep though.

How did you solve the "6C" Q-Code issue btw?

Quote:


> Originally Posted by *squidman*
> 
> Hi Sorry new topic (link aggregation would be great for me too! but not much point as am only on 50 down/10up subscription...maybe iff I get around to building a Norco 4224 storage server)....
> 
> OK actually 2 questions (on z9pe-d8 ws with 2x QA8X e5-2665):
> 
> 1) I read somewhere that it was possible to force the turbo to max on all cores, but haven't figured out the bios settings...did successfully dial the bclk up a tiny notch or two, but bit pointless so set it back to automatic....rendering at the moment my 2665's pretty much max out at 2.4ghz...which still sucks for e.g. Vue Xstream or stabilizing with Mercalli...SLOOOW....on the consumer boards the feature was called ASUS something or other....
> 
> 2) Has anyone got the IRSTe software to work? Have 4 WD red 4tb in raid 5 on the bottom 4 SATA/SCU ports, no probs with the intel raid functionality in BIOS, but the IRSTe software always crashes. The marvell web-based gui works fine for me (ssd caching). Should i uninstall (hoping that won't mess with my raid...would be catastrophic) and start from scratch? Wondering whether I have to reset the OPROM settings in BIOS. In fact I'm a bit confused...is the bios IRSTe OPROM different and separate from the Windows IRSTe utility? I thought the latter was just a windows gui (like the marvell web-based utility) to monitor the raid set....
> 
> Thanks! Good to be back, will have more questions on Taobaoing some e5-2695/7's in a bit in a few posts (NEED cinebench 11.5 score of 34!)....
> 
> Shame that Algeria didn't win...they would have been facing france in the quarter finals the day after Algerian independence day, a chance to reallly stick it to them!


Not sure about forcing it to max boost constantly, but I have mine at 24x 104MHz bClk = 2.5GHz. It turbos up to 2.6-2.7GHz on single threaded work though.

Quote:


> Originally Posted by *drjam*
> 
> Thanks for that reply! It answers my questions.
> Funny this isnt posted on the official site somewhere.
> A digital high-five is coming your way man!


No problem, I think you can find that info on the internet but only if you search for "Registered RAM Density". Not always that easy to find 

Anyways, when you look at registered RAM you have a Register Chip typically in the center of the PCB (this particular one uses an Inphi register)










Instead of addressing the memory chips directly, the memory controller on the CPU can address the register. Because of that lower load on the memory controller you can have more chips per module (higher density).


----------



## Stefan7

So I finally got all the parts together for my build. I looked at the sockets on the mother board and the first socket has 4 bentup pins!!!! god damn it


----------



## ccRicers

Quote:


> Originally Posted by *Stefan7*
> 
> So I finally got all the parts together for my build. I looked at the sockets on the mother board and the first socket has 4 bentup pins!!!! god damn it


If you bought this new/with a guarantee, try to return it for a replacement. If it doesn't come with a warranty, you can make a deal with the seller, but I would try to fix the pins myself. Four isn't a lot. I had to fix one bent pin for my CPU because the discrete GPU wasn't overriding the iGPU (core i5 chip). Using the tip of a clean x-acto knife you can bend it carefully and gently to the direction that the other pins are facing.


----------



## squidman

Back to ES Taobaoing...

Does this look like a decent deal? C0 and a bit slow on all cores....but seems ok for 6200 yuan..however seems almost too good to be true.....http://item.taobao.com/item.htm?spm=a230r.1.14.15.ogaPhA&id=24986968352&ns=1#detail

Others...http://item.taobao.com/item.htm?spm=a230r.1.14.70.ogaPhA&id=39196344488&ns=1#detail for a lot more dough....

Or do I go an e5-2695 v2? or even a 10-core with better stepping (higher turbo with all cores going?)....or even an older v1 e5? Dunno, but at the moment my e5-2665's are just barely higher (cinebench of 17) than a normal oc'd i7, which would be fine but they are a bit too slow stabilizing vids, can just forget about vue xstream, and ok with mental ray and vray, but not exactly blazing...and will be doing loads of rendering from the fall on (going back to school doing 3d design and animation). Haven't tried Arnold yet, but then am a 3ds user (school uses maya, so there ya go).

My budget for the upgrade is pretty much a max of 6k yuan per cpu (grand usd per cpu), preferably less....and then prbly have to fork out for ecc memory (although a lot of the taobao screenshots are showing non-ecc memory for these ES chips)....

Suggestions?


----------



## TheBlademaster01

I just got some E5-4650s for $250 a piece (C0).

If you can afford them, E5-2680V2 (QD29) and E5-2695V2 are best value (around 5k yuan). Obviously the 2680V2 will give you better per core performance though. I'm hoping ivy prices will drop when Haswell 18 cores drop


----------



## squidman

Cool thanks for the heads up! Will have to pay off the school first unfortunately, but maybe q4 I can go shopping...









Edit: bit unrelated, but back in the day when I stayed on Prinsengracht, I went to a very cool part of town (bohemian) with a market etc, and the best local witbier I've ever had. What are the famous witbiers brewed locally in Amsterdam? Homebrewing here, have just started 50l, but would like to emulate the summer beers of Amsterdam (from Norway obviously, stupid prices...110 kroner+ for a specialty beer or 15 euro!!!)...

Thanks...

Edit...could have been rosengracht...in any case a very hip local market, hip place with lots of local brews, students etc....never mind (lots of places like that! very cool anyway! what a great town...)...


----------



## squidman

On our board, think the e5-26xx are the last of the line. There are E7's of course (and have seen on the interweb a few supermicro systems with 4 e.g. puget systems), but pretty sure for the z9pe-d8 ws series (or just the intel c600 chipset) Ivy v2 xeons are the last of the line. So top shelf being the e5-2697v2, then the e5-2695v2, then others like you mentioned which may or not be better for rendering depending on maximum average turbo....

Of course if you really really need to render a la pixar at 4k then I guess no choice but to do that off-line. For the rest of us mortals, dialling down the settings a bit, doubling rendering capacity by going to say 2695v2 seems pretty good. Especially iff you can taobao it with ES versions...(but the elephant is of course needing ECC ram..maybe? maybe not? pretty expensive regardless)...

happy rendering peps!


----------



## Devione

Hi there guys,

This place seems to be the most comprehensive and helpful not to mention knowledgeable for this rather niche product.

I was just wondering if this product would still be worth buying seeing as Broadwell-EP, which assuming I am correct, is the successor for this chipset, is just around the corner.

Also, is there any word of Asus perhaps releasing a similar product but with Broadwell-EP instead?

Would love to hear from you guys. Thanks for your time.


----------



## TheBlademaster01

Quote:


> Originally Posted by *squidman*
> 
> Cool thanks for the heads up! Will have to pay off the school first unfortunately, but maybe q4 I can go shopping...
> 
> 
> 
> 
> 
> 
> 
> 
> 
> Edit: bit unrelated, but back in the day when I stayed on Prinsengracht, I went to a very cool part of town (bohemian) with a market etc, and the best local witbier I've ever had. What are the famous witbiers brewed locally in Amsterdam? Homebrewing here, have just started 50l, but would like to emulate the summer beers of Amsterdam (from Norway obviously, stupid prices...110 kroner+ for a specialty beer or 15 euro!!!)...
> 
> Thanks...
> 
> Edit...could have been rosengracht...in any case a very hip local market, hip place with lots of local brews, students etc....never mind (lots of places like that! very cool anyway! what a great town...)...


Can't help you much there since I don't drink .

Quote:


> Originally Posted by *squidman*
> 
> On our board, think the e5-26xx are the last of the line. There are E7's of course (and have seen on the interweb a few supermicro systems with 4 e.g. puget systems), but pretty sure for the z9pe-d8 ws series (or just the intel c600 chipset) Ivy v2 xeons are the last of the line. So top shelf being the e5-2697v2, then the e5-2695v2, then others like you mentioned which may or not be better for rendering depending on maximum average turbo....
> 
> Of course if you really really need to render a la pixar at 4k then I guess no choice but to do that off-line. For the rest of us mortals, dialling down the settings a bit, doubling rendering capacity by going to say 2695v2 seems pretty good. Especially iff you can taobao it with ES versions...(but the elephant is of course needing ECC ram..maybe? maybe not? pretty expensive regardless)...
> 
> happy rendering peps!


Yes, for this board E5-2697V2 is end of the line unfortunately. Sandy Bridge chips are considerably cheaper. Sometimes you find some great deals on ebay as well.

Quote:


> Originally Posted by *Devione*
> 
> Hi there guys,
> 
> This place seems to be the most comprehensive and helpful not to mention knowledgeable for this rather niche product.
> 
> I was just wondering if this product would still be worth buying seeing as Broadwell-EP, which assuming I am correct, is the successor for this chipset, is just around the corner.
> 
> Also, is there any word of Asus perhaps releasing a similar product but with Broadwell-EP instead?
> 
> Would love to hear from you guys. Thanks for your time.


Yes, Asus will release the Z10PE-D16










Not sure if they'll do a WS board again, since the Z9PE-D8 WS was designed under the assumption that Sandy Bridge would be able to OC. Now Asus knows better I guess...

I think it's better to sit this one out and watch how the market will develop. DDR4 prices will be high and so will the 18 and 16(?) core chips. Maybe something like 10/12 core chips will be good value this time around. Difficult to say at this point in time really. I'm not in a hurry, Intel platforms tend to keep much of their value even if it's not bleeding edge technology anymore.


----------



## Devione

Quote:


> Originally Posted by *TheBlademaster01*
> 
> Can't help you much there since I don't drink
> 
> 
> 
> 
> 
> 
> 
> .
> 
> Yes, for this board E5-2697V2 is end of the line unfortunately. Sandy Bridge chips are considerably cheaper. Sometimes you find some great deals on ebay as well.
> 
> Yes, Asus will release the Z10PE-D16
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> Not sure if they'll do a WS board again, since the Z9PE-D8 WS was designed under the assumption that Sandy Bridge would be able to OC. Now Asus knows better I guess...
> 
> I think it's better to sit this one out and watch how the market will develop. DDR4 prices will be high and so will the 18 and 16(?) core chips. Maybe something like 10/12 core chips will be good value this time around. Difficult to say at this point in time really. I'm not in a hurry, Intel platforms tend to keep much of their value even if it's not bleeding edge technology anymore.


Thanks for your quick reply. A little Googling about the Z10 seems to have lead me to this

http://www.anandtech.com/show/8148/computex-2014-the-asus-booth-tour/6

and there appears to be a WS E-ATX variant, however it seems much more server oriented compared to the current Z9, which is a shame.

Thanks again however for your advice.


----------



## TheBlademaster01

No problem.

Those 2P boards are the Z10PE-D16 and the Z10PA-D8. The latter is ATX, not E-ATX. Seems to be a direct successor to the Z9PA-D8. The Z10PE-D16 seems to be more of a successor to the Z9PE-D16 but that one was fairly similar to the Z9PE-D8 WS to begin with (if not for the overclocking options and additional PCI-E lanes)


----------



## Stefan7

cc thanks for the reply I tried the bend the pins back method, it got them back, then i decided that one looked different coloured so thought this must of been an alignment issue, further fiddling and you guessed it, it snapped!..ouch.....The miss colouration was down to i think it having a bend/fracture line. So it just got progressively worse and i watched it deteriorate into a brake in front of my eyes. Should of left it!...but then again would it of snapped when i needed most it at a later date? so any way..you live and learn. the board was second hand..no warranty, and pretty sure i bent the pins my self by accident..

Any ways I'm getting impatient to build this damn thing now. And abit of searching around and i found another board that had, had a socket refurb from asus. But this time with a golden 90 day warranty. @£240 - Much cheaper than £450+ new which i frankly cant afford. I'm awaiting this to arrive in the post.

Meanwhile I'm going to try and send this back to asus and get a new socket put on it. If poss then sell to recoup some of my loss.....

*anyone any idea how much it will cost for a new socket. Guessing there will be a fair ammount for p&p from UK ot netherlands.*

Cheers Folks


----------



## Stefan7

all this to render some crappy architecture projects in revit because it doesnt support GPU rendering.. well I suppose it is my choosen career path and i make a living dossing about on a computer


----------



## TheBlademaster01

Wasn't there an Asus repair center in Benelux? Either way, shipping to UK would be somewhere along the lines of 15-20 euros.

Also, I installed my 4650s today and all of my posting, memory and GPU issues are gone









Can't seem to overclock it though...


----------



## squidman

Hi sorry new topic again,

Have had numerous BSOD's lately, or just plain shutdowns, and my win 7 64 ultimate seems to be pretty corrupted. Have tried system restore to a month back, command-line sfc etc to no avail. Am wondering whether anyone with this board has upgraded from win 7 to 8.1? The main thing for me, is that I have a raid5 on on the SCU sata/sas ports (no the intel RSTe GUI never worked for me either! but the oprom on boot seems to be working!), how could that be affected by performing an 'upgrade' from a win 8.1 dvd? Have 8? tb of data on that raid setup, the rest is jbod so should be pretty secure (apart from the Marvell ssd caching on sata 6-10!).

Has anyone tried upgrading, or have experience with win 8.1`? Or should I go server OS? Bit fed up with win7 at the moment, but obviously reinstalling is a monumental pain in the a$$, not to mention upgrading. Have OS and programfiles on separate ssd's at the moment, so no can do the standard win7 from dvd upgrade thing.

Win 8.1 mostly as win7 is no longer available retail here in Norway (can't repair from OEM dvd's) and because, well a 4k screen is almost inevitable in the near future (the 28 inch AOC looking very nice at a nice price of 4k NOK or 378 GBP or 648 USD)....

Don't think it is memory or other things at the moment (could be broken corsair link!), most likely corrupted win7....

Oh yes...for 2p board like ours, which is the best version of 8.1 to install?


----------



## TheBlademaster01

BSODs are apparently also a problem of B0 chips and the .CAP BIOSes. Linux locks up less than Windows from my experience. After a long period of lock ups it seems that these C0 chips fixed the problem.

I have no experience with Windows 8.1 and this system though


----------



## Stefan7

Ok got the the new (refurbished) board. Got it up and running after several attempts and it boot looping alot and found that it was having issues with bios and quadro k4000 card. There is a fix in a late version of bios, but ive heard that later bios's stop the support of e5-2665's for some bizzare reason!..told this by the guy who i bought the chips and board off, so dont want to flash it and then have to buy another bios chip like he did...


----------



## TheBlademaster01

Best is to run the 2665s on BIOS 0503/0703 with a GTX 580 or something older. My 2665s posted on the latest BIOS version, but I wouldn't recommend it. You get all kind of weird RAM and GPU issues and system lockups. Only solution is to buy a new BIOS chip with 0503 at that point.

Are your 2665s QA8X/QA8Z chips?

Also, I get the boot looping as well. There was a fix for that, but it involved disabling the Marvell controller IIRC.


----------



## Stefan7

i got it flashed upto vr 3506 which supports a k5000, and it some how it works the k4000 too in lane 2 but not in lane 1 at all!, however its intermittent on boot sometimes posting sometimes not. I read earlier in this post that it doesnt always allocate the specified memory amount on post from bios.... I'm wary to flash to vr 5404 for full k4000 as this is the second newest bios and im not liking what im hearing about the newer bios problems with these chips, guessing they are pretty old now then?...i will have to check the exact vrs of the chips i have, not to sure. Not sure how to proceed, sell my quadro k4000?...


----------



## TheBlademaster01

3302 is better than 3506. For some reason 3506 disabled quad channel for me.

Also, by flashing to 3506 the system will now be unstable with B0 chips. You can run the system but it may randomly lock up etc.

You can get it to POST 1 out of 3-5 times on cold boots though, and 3302 and later is needed for Kepler and newer GPUs.


----------



## Stefan7

so its new chip time already, then









thinking of bailing on this board, as asus cant seem to sort the bios's out right, why would the chips become unstable on newer bios's thought the idea was to add more hardware support not take it away!


----------



## Stefan7

is there a way to check compatiblity with bios and chips? there is little/no info on asus's site


----------



## Stefan7

hp z820 motherboard anyone? haha


----------



## TheBlademaster01

Quote:


> Originally Posted by *TheBlademaster01*
> 
> Z9PE-D8WSCompatibility.pdf 73k .pdf file
> 
> 
> Here's a PDF in case you can't read it
> 
> 
> 
> 
> 
> 
> 
> 
> 
> E:
> 
> Yes, QB7R are pretty much retail E5-2687W. You should however make sure 5404 works with the other peripherals. Asus is known to brick things in the process (think storage, VGA etc.). I have no data on that.


See above









Quote:


> Originally Posted by *Stefan7*
> 
> so its new chip time already, then
> 
> 
> 
> 
> 
> 
> 
> 
> 
> thinking of bailing on this board, as asus cant seem to sort the bios's out right, why would the chips become unstable on newer bios's thought the idea was to add more hardware support not take it away!


It's because Sandy B0 never worked correctly with Kepler etc. Also, they drop early ES support the first moment. Not sure if Supermicro also has this problem.

You should really only use C stepping CPUs on this board. Ivy B1/B2 have been fine until now though.


----------



## Stefan7

ouch - B0 Up to 3109 only then - alot of hasle and problems.

possibly 2 x Z9PE-D8-WS For sale shortly


----------



## Stefan7

I dont suppose you know if a HP Z820 board would be any better, i have seen quite alot of them with 2665's and k4000 ?..so guessing so...thanks for you help btw









No possible way to manually code bios then?...my dad is a programmer knows abit about this, but this might be out of his league saying that!....said that its probably an epromm and that you use to be able to put them under ultraviolet light to reset bios lol not sure how much of what he is saying is true.


----------



## TheBlademaster01

Maybe Supermicro X9DA7/X9DAi?

I don't think E5-2665 is a problem, just B0 2665s, 2687W or any B0 Sandy for that matter.

You could also try swapping the BIOS chips of your 2 boards if the other board has 0503 or 0703 and don't update past that. You can use the system with GTX 580 and older then.


----------



## Stefan7

I Think i know what your saying the B0's are engineering samples, and this is probably the reason i.e they want to stop compatibility asap!


----------



## TheBlademaster01

No, C0/C1 are ES as well but they are pretty polished already. Sandy B0 just has too many bugs.


----------



## Stefan7

blurg!


----------



## TheBlademaster01

Yeah I've got a pair of 2665s over here as well. Coped with them for like 6 months on v3302 and 780Ti. A lot of headaches, but still usable to some extent. Just last week I got a pair of C0 E5-4650s for $500. Felt comforting to see all the problems go away. GPUs POST in any of the slots, no lockups etc.


----------



## squidman

There was a nice chart of Bios and ES version compatibility a fair ways back in the thread (was it Blademaster who made it?), could have referred only to sandy v1 chips tho'. Also hadd ecc/non-ecc ram compatibility (although I've seen a lot of taobao screenshots of b0 ivy bridge v2's working with non-ecc...)...

Not had any problem with my b0 e5-2665's until recently, after doing some stupid stupid messing with bios settings and installing the latest windows updates (june july 2014) on an already flaky win7 sp1 64 bit. Was on a .cap bios as well (3506?) which actually was stable for at least 6 months! Just had to go tweaking, stupid me.....
















Edit: meant ivy xeon v2 chips of course...


----------



## TheBlademaster01

IVB-EP only needs ECC to POST on v5004







. It would be better if Asus removed that revision from their website though...

Also, Sandy B0 is never really stable on .CAP. I've crunched work for 4 days 24/7 on the system and it was fine. Other days it would fail in a matter of hours.


----------



## Stefan7

cheers fella, yeah blademaster posted me the Bios PDF im currently on 3506 release so you've given me abit of faith to install windows 7 and give it a try, i can put up with the only booting up 2 - 5 times if it runs stable enough. Otherwise i think il swop my graphics card.

since ive calmed down over the weekend, can anyone give me some info on this tabboaing or whatever its called, just a brief outline of what it even is?...

Also are there any xeons with unlocked multipliers for overclocking. I wasnt aware of any but i saw a youtube video with someone running dual hexcore x5660's @ 4ghz

thanks folks


----------



## Stefan7

Ah just been reading about the x5660's on another forum, apparently its to do with an uncore whatever that is and it unlocks on a late version of bios. Come on asus give us an unlocking bios for our chips









I think the HP Z820 Has the same issues with B0 stepping

Back to the build
I think i will down grade my quadro K4000 to an older 4000 without keplar architecture and roll back the bios/ get a new bios chip, on your advise blademaster if this set up is totally unstable.


----------



## TheBlademaster01

You need a BIOS chip with version 0503 first. I don't know if Quadro 4000 is supported though. I know the GeForce variant is







.

And squidman means taobao. A chinese marketplace similar to eBay. They have a lot more and often cheaper ES chips.


----------



## Stefan7

Thanks!


----------



## Stefan7

the quadro 4000 is a 2.0 pci-e - would this help?


----------



## squidman

Long boot loops could be due to how you set the PXE settings in bios...i allow it to run a while as the board is going through all these checks anyway and i have 2 raid systems but there is an option to disable pxe looping. Havent tried it yet.


----------



## 82ghost82

Hi!
I noticed some problems with my ram.
I have a system of 2 workstations connected by infiniband cards.
First workstation has 64 gb samsung ecc ram installed (at 1333 Mhz), the second one has 32 gb samsung ecc ram (at 1333 Mhz).
These works in parallel for computational fluid dynamics.
After some time I noticed that my system slow down a lot and after some tests I noticed that the problem is the ram (only in the 1st workstation with 64gb ram).

Maxxmem test shows:
Usual operation:
Memory copy: 7566 Mbytes/s, Memory Read: 18924 Mbytes/s, Memory write: 6157 Mbytes/s, Memory latency: 76.5 ns

When the system slows down:
Memory copy: 489 Mbytes/s, Memory Read: 532 Mbytes/s, Memory write: 5460 Mbytes/s, Memory latency: 150.0 ns

In both case it is reported that memory works in quad channel at 1333 Mhz.
It seems latency, copying and reading are slowing down my system







(writing seems is not affected..).

Anybody knows what can be the problem?

Thanks

PS: testing by disabling patrol scrub and demand scrub to see if it's better...
No effect
Thermal throttling: no effect

Now resetting to default settings: *OK, it seems all runs smooth now...*


----------



## Stefan7

https://www.asus.com/uk/support/Article/580/

A Link for repairs for workstations with as asus put it a 'Customer-Induced Defect' or as i put it a clumsy hand trying to re bend socket pins!

Still havent got anywhere with them mind!


----------



## LeMakisar

Hello everyone !

I'd like to know for SLI systems, if it's alright to put card on any slots (1, 3, 5 or 7) ?

I have a PCIe 1x sound card and a 8x controller card. I'm thinking of the following configurations:

- PCIe1 : sound card
- PCIe3 : GPU 1
- PCIe 5 : controller card
- PCIe 7 : GPU2

OR inverting GPU1 and controller card, if slots 3/7 are not OK for SLI.

I use 2 retail CPUs, if it makes any difference

Another question : if I use slots 5 and 7, will an EVGA short bridge be OK ?

Thanks !


----------



## Big Jim

I sent in the Motherboard to Asus for repair. No word other than it is going to cost $437.00 for the repair. Some how one of the CPU mounts was discovered to have bent pins, even though covers were in place for shipping protection. Funny that Asus had motherboard for 2 weeks before my calling them to check status and they never bothered to report bent pins. Looking at 5 weeks of down time and buying the motherboard twice with this repair bill added to original cost. No word on what caused the bios to fail or what the Q-code fix was or if it will even was addressed. No answers for the correct bios or settings needed for my components. Just give Asus $437.00 within 4 days or the motherboard would be returned unrepaired. No other information was given. It took Asus 2 weeks to report this to me. Now, it will take another week for the repair and a second for the shipping it back to me. 5 weeks for what Asus Support said would be a 7 day turn around. I get no answers as to the bios issue or why the pins were bent, even though they were covered. Not very happy at this point.


----------



## Stefan7

Quote:


> Originally Posted by *Big Jim*
> 
> I sent in the Motherboard to Asus for repair. No word other than it is going to cost $437.00 for the repair. Some how one of the CPU mounts was discovered to have bent pins, even though covers were in place for shipping protection. Funny that Asus had motherboard for 2 weeks before my calling them to check status and they never bothered to report bent pins. Looking at 5 weeks of down time and buying the motherboard twice with this repair bill added to original cost. No word on what caused the bios to fail or what the Q-code fix was or if it will even was addressed. No answers for the correct bios or settings needed for my components. Just give Asus $437.00 within 4 days or the motherboard would be returned unrepaired. No other information was given. It took Asus 2 weeks to report this to me. Now, it will take another week for the repair and a second for the shipping it back to me. 5 weeks for what Asus Support said would be a 7 day turn around. I get no answers as to the bios issue or why the pins were bent, even though they were covered. Not very happy at this point.


i beg to differ asus repair serivce is abosolutely top drawer! hahaha














i cant even get a quote off them mate!....im going to try elsewhere or attempt to fix this broken pin myself.

Jim i wonder if you could take them to court over the timescale that they gave you, if its a computer you use for work it could count as loss of earnings

So far this build has been the most expensive/biggest waste of time ive been involved with lol

I have one of the two boards i purchased working now, in a fashion. 1 - 5 TIMES it will boot on pressing the on button, because the bios only supports B0 stepping processors to a certain point before it drops them or starts playing silly buggers, because the bios is wrote in favour of C steppers onwards..... Been reading about Hp Z820s ws's they seem to have the same probs....I'm guessing the majority of large companies are the same though in there give me money and run policies?...anyone had any good experiences with other companies out there?
*
I dont think they even have an automated service that emails you when they have replied to a message on their site...so you have to log in and check every other minute....come on this is pretty standard stuff to have on a website nowadays*


----------



## TheBlademaster01

Quote:


> Originally Posted by *Stefan7*
> 
> the quadro 4000 is a 2.0 pci-e - would this help?


Not really. I would assume that it works, but I've heard people complain about Quadro 6000 before. I know for certain that the GeForce variants work, haven't properly tested or seen Fermi Quadros before. (Quadro K5000 is also PCI-E 2.0 and doesn't work)

Quote:


> Originally Posted by *squidman*
> 
> Long boot loops could be due to how you set the PXE settings in bios...i allow it to run a while as the board is going through all these checks anyway and i have 2 raid systems but there is an option to disable pxe looping. Havent tried it yet.


I forgot about that, thanks for mentioning it 

Quote:


> Originally Posted by *82ghost82*
> 
> Hi!
> I noticed some problems with my ram.
> I have a system of 2 workstations connected by infiniband cards.
> First workstation has 64 gb samsung ecc ram installed (at 1333 Mhz), the second one has 32 gb samsung ecc ram (at 1333 Mhz).
> These works in parallel for computational fluid dynamics.
> After some time I noticed that my system slow down a lot and after some tests I noticed that the problem is the ram (only in the 1st workstation with 64gb ram).
> 
> Maxxmem test shows:
> Usual operation:
> Memory copy: 7566 Mbytes/s, Memory Read: 18924 Mbytes/s, Memory write: 6157 Mbytes/s, Memory latency: 76.5 ns
> 
> When the system slows down:
> Memory copy: 489 Mbytes/s, Memory Read: 532 Mbytes/s, Memory write: 5460 Mbytes/s, Memory latency: 150.0 ns
> 
> In both case it is reported that memory works in quad channel at 1333 Mhz.
> It seems latency, copying and reading are slowing down my system
> 
> 
> 
> 
> 
> 
> 
> (writing seems is not affected..).
> 
> Anybody knows what can be the problem?
> 
> Thanks
> 
> PS: testing by disabling patrol scrub and demand scrub to see if it's better...
> No effect
> Thermal throttling: no effect
> 
> Now resetting to default settings: *OK, it seems all runs smooth now...*


Yes, it seems like some kind of instability (error correction kicking in?)

Quote:


> Originally Posted by *LeMakisar*
> 
> Hello everyone !
> 
> I'd like to know for SLI systems, if it's alright to put card on any slots (1, 3, 5 or 7) ?
> 
> I have a PCIe 1x sound card and a 8x controller card. I'm thinking of the following configurations:
> 
> - PCIe1 : sound card
> - PCIe3 : GPU 1
> - PCIe 5 : controller card
> - PCIe 7 : GPU2
> 
> OR inverting GPU1 and controller card, if slots 3/7 are not OK for SLI.
> 
> I use 2 retail CPUs, if it makes any difference
> 
> Another question : if I use slots 5 and 7, will an EVGA short bridge be OK ?
> 
> Thanks !


It's best to connect both GPUs to the same CPU for minimal latency (though the SLi bridge would most likely provide sufficient bandwidth to prevent using the slow PCI-E link)

Short bridge should be okay for SLI in [#1 & #3] OR [#3 & #5] OR [#5 & #7] configuration. Otherwise a long bridge is necessary.

Quote:


> Originally Posted by *Big Jim*
> 
> I sent in the Motherboard to Asus for repair. No word other than it is going to cost $437.00 for the repair. Some how one of the CPU mounts was discovered to have bent pins, even though covers were in place for shipping protection. Funny that Asus had motherboard for 2 weeks before my calling them to check status and they never bothered to report bent pins. Looking at 5 weeks of down time and buying the motherboard twice with this repair bill added to original cost. No word on what caused the bios to fail or what the Q-code fix was or if it will even was addressed. No answers for the correct bios or settings needed for my components. Just give Asus $437.00 within 4 days or the motherboard would be returned unrepaired. No other information was given. It took Asus 2 weeks to report this to me. Now, it will take another week for the repair and a second for the shipping it back to me. 5 weeks for what Asus Support said would be a 7 day turn around. I get no answers as to the bios issue or why the pins were bent, even though they were covered. Not very happy at this point.


That sounds almost as bad as Gigabyte, although they actually solved the problem without costs (assuming no end user errors have been found). They didn't notify anything about how they fixed it however, and the contact persons I wrote were pretty bad at English.

Things like this really encouraged me to buy a working board used. Too many things that could go wrong with this board (especially when using ES chips).

Quote:


> Originally Posted by *Stefan7*
> 
> i beg to differ asus repair serivce is abosolutely top drawer! hahaha
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> i cant even get a quote off them mate!....im going to try elsewhere or attempt to fix this broken pin myself.
> 
> Jim i wonder if you could take them to court over the timescale that they gave you, if its a computer you use for work it could count as loss of earnings
> 
> So far this build has been the most expensive/biggest waste of time ive been involved with lol
> 
> I have one of the two boards i purchased working now, in a fashion. 1 - 5 TIMES it will boot on pressing the on button, because the bios only supports B0 stepping processors to a certain point before it drops them or starts playing silly buggers, because the bios is wrote in favour of C steppers onwards..... Been reading about Hp Z820s ws's they seem to have the same probs....I'm guessing the majority of large companies are the same though in there give me money and run policies?...anyone had any good experiences with other companies out there?
> 
> *I dont think they even have an automated service that emails you when they have replied to a message on their site...so you have to log in and check every other minute....come on this is pretty standard stuff to have on a website nowadays*


Also depends on where you reside.

I heard good things about EVGA. Personally I have had an excellent experience with Koolance and Mayhems (watercooling though).


----------



## 82ghost82

About ram problems: also resetting to default settings initially helped but after some time latency from about 75 ns rised to 150+ ns








I think the problem could be the voltage: 1 h running and till now no decrease in performance; I manually increased psa voltages to 0,980 V and ram sticks to 1,55 V and returned to standard 1333 Mhz (auto).
It seems when the system is at full load treshold is reached and no sufficient power is delivered with auto settings (??).
Will update the thread.


----------



## LeMakisar

Quote:


> Originally Posted by *TheBlademaster01*
> 
> Not really. I would assume that it works, but I've heard people complain about Quadro 6000 before. I know for certain that the GeForce variants work, haven't properly tested or seen Fermi Quadros before. (Quadro K5000 is also PCI-E 2.0 and doesn't work)
> 
> I forgot about that, thanks for mentioning it
> 
> 
> 
> 
> 
> 
> 
> 
> 
> Yes, it seems like some kind of instability (error correction kicking in?)
> 
> It's best to connect both GPUs to the same CPU for minimal latency (though the SLi bridge would most likely provide sufficient bandwidth to prevent using the slow PCI-E link)
> 
> Short bridge should be okay for SLI in [#1 & #3] OR [#3 & #5] OR [#5 & #7] configuration. Otherwise a long bridge is necessary.
> 
> That sounds almost as bad as Gigabyte, although they actually solved the problem without costs (assuming no end user errors have been found). They didn't notify anything about how they fixed it however, and the contact persons I wrote were pretty bad at English.
> 
> Things like this really encouraged me to buy a working board used. Too many things that could go wrong with this board (especially when using ES chips).
> 
> Also depends on where you reside.
> 
> I heard good things about EVGA. Personally I have had an excellent experience with Koolance and Mayhems (watercooling though).


thanks, I'll try 5 and 7 then, it'sthe same CPU


----------



## LeMakisar

arg, if I us PCIe 7 Ill loose my front USB 2 and UBS3, and PCIe1 isn't OK to plug a card because of a heatsink just below CPU1 :/
I he latency you mentioned that problematic if I use 3 and 5 ?
Woul I loose like &fps or more like 10 ?


----------



## TheBlademaster01

You could get stutter in games (so periods of 0 FPS and then back to nominal). I know for certain Crossfire Hawaii cards have this problem. I'm not 100% sure the same will happen with nVidia because it has the SLI bridge for communication.


----------



## LeMakisar

ok I'll try 5 and 7 then, USB2 is OK, that's only USB3 I can't plug ;..

One thing I didn't realize : cards are VERY close to each other, I can't imagine how the upper card will suffer from heat


----------



## Stefan7

Quote:


> Originally Posted by *TheBlademaster01*
> 
> I just got some E5-4650s for $250 a piece (C0).
> 
> If you can afford them, E5-2680V2 (QD29) and E5-2695V2 are best value (around 5k yuan). Obviously the 2680V2 will give you better per core performance though. I'm hoping ivy prices will drop when Haswell 18 cores drop


was you bidding on these on taobao?....seems pretty cheap, cheaper than what i can find on it

i have a friend who has been learning japanese for the past 3 years, kanji, hiragana & katakana is pretty much chinese symbols I think, so i will put him to the test. See how much he can read on this taobao thing

I went to japan afew years ago too, yen and yuan are very different in currency rates ha!


----------



## TheBlademaster01

I got them from ebay (Europe). The person I bought them from couldn't get them to work in his server.

I can read Japanese, not Chinese. But I just guess a bit while browsing. For buying I'd advise an agent like bhiner.


----------



## Stefan7

a E5-2697v2 ES cheapest on there is 5850.00 yuan around about £551.69 - but not sure if this is buy it now or bidding.


----------



## Stefan7

Quote:


> Originally Posted by *TheBlademaster01*
> 
> I got them from ebay (Europe). The person I bought them from couldn't get them to work in his server.
> 
> I can read Japanese, not Chinese. But I just guess a bit while browsing. For buying I'd advise an agent like bhiner.


Thanks for the good advise dude, my friend said japanese and chinese arent the same to read. I managed to get chrome to translate the pages. You have to translate each page as you browse. I think i got quite an alright bargin at £200 a chip for e5 -2665 even though they are early B0 Stepping and Es's. They arent that common on the english version of ebay.

oh yes and there are some quite ammusing translations into english when on taobao - buy now at cheap price whisper, do not hesitate to seize the start!


----------



## Stefan7

theres some C0 stepping E5-2665's on taobao for £236.05 each. and some E5-4650's for £254.94 Im guessing setting up and an account will be tricky


----------



## ZDngrfld

Quote:


> Originally Posted by *Stefan7*
> 
> Im guessing setting up and an account will be tricky


www.bhiner.com


----------



## Stefan7

cheers


----------



## Stefan7

Back to the broke pin on the other board i have, I just received an email response really dont know what to do now as i have heard so many horror stories about asus rma. The charge of 58.79 seems reasonable, im not sure if thats both the repair cost and shipping cost, its not worded that well. I'm guessing its not. In the mean time i found someone who reballs and flows xbox/ gpu chips, and have sent off for a socket base from china which was £10 including shipping. He said he would have a look and see if was possible.

Im guesssing ill send it off to asus and then get an emailing back once they have it saying its going to cost x ammount more after we've looked at it, or its irrepairable?

I think ill wait for the socket base and then try the local route at least it will only cost £10 if he says he cant even attempt it.

Email

[email protected]
To Me,[email protected]

Today at 2:41 PM

Good afternoon,

We are sorry for delayed answer for you case. I am sorry, I have no information about resocketing machine in Benelux. This is Asus Czech Service located in Czech Republic. We can try to repair your motherboard. If repair is successful, so we will charge the repair and shipping costs (Price EUR 58,79 ex. VAT). Please note that there is also a chance that MB is irreparable. If irreparable, then we will send the MB back to you free of charge.

If you want to repair the MB, so please provide me serial number of the MB and full address where to pick it up and delivery it after repair. Also provide me contact information (name, phone, email).

Thank you

With kind regards

Advanced Replacement Service

ASUS Czech Service

Tel.: +420 596 766 891
Fax: +420 596 766 329
Email: [email protected]


----------



## TheBlademaster01

You might want to delete the sender's name.

But seriously, that's horrible. Might as well try locally if this is the only option...


----------



## Stefan7

done

its very much like a ransom note haha


----------



## TheBlademaster01

I would have thought Asus had better service on these motherboards...


----------



## anubis1127

Quote:


> Originally Posted by *TheBlademaster01*
> 
> I would have thought Asus had better service on these motherboards...


Sarcastic TheBlademaster01 is sarcastic.


----------



## TheBlademaster01




----------



## LeMakisar

Any info on when to get a Z9PE before it's not sold anymore ?


----------



## XZSAMURAI

*Hi Guys

Anyone can tell me Does this mainboard support Corsair Dominator Platinum 64GB 2133MHZ Without any problem ?*


----------



## anubis1127

Quote:


> Originally Posted by *XZSAMURAI*
> 
> *Hi Guys
> 
> Anyone can tell me Does this mainboard support Corsair Dominator Platinum 64GB 2133MHZ Without any problem ?*


I was only able to run 1600Mhz when I had a z9pe-d8, no matter what the RAM spec. I was using SandyBridge E5 2680s though, with v2 chips I do not know.


----------



## TheBlademaster01

I have 32GB Corsair Dominator Platinum 1866MHz in my system. I can run 1866MHz by OC'ing but it is unstable in the long run. 1600MHz is what you should expect. Ivy IMC is better, but other than ECC and registered RAM I haven't seen anyone do 1866MHz or higher.

I haven't played with Ivy chips yet though.


----------



## XZSAMURAI

i living in dubai i search in local store , right now platinum is only choice , i dont know what to do , i just want to work with my mainboard without any problem that's enough for me , im very confused about ram , in here around 1100 US dollar i should to pay for this ram i don't want make any mistake


----------



## TheBlademaster01

It will work, but only at 1600MHz speed. So if you can save some money on frequency you should do it.


----------



## Stefan7

Socket arrived lets see whos brave / stupid enough to take on the swop


----------



## TheBlademaster01

Lemme know how it goes lol


----------



## squidman

Dude is replacing the sockets? Wow he has ones that hang low, give him that much! Didn't even think it was possible, but I guess they have to make it simple for the slaves over there in manufacturing!


----------



## Saieash

Quote:


> Originally Posted by *ironcurtain*
> 
> I previously fixed this somehow, now it's back to non/operational....I simply rebooted after updaing the BIOS to the latest version as of today. It's once again not posting. I cleared CMOS and plugged a Kingston 16GB drive with the converted BIOS, and this didn't help. I haven't taken out the CPUs or DIMMs yet, because this would be the third time I go through all the mess.
> 
> What can I do?


I seem to be in the exact same predicament you are in and I was just wondering how you got it working.
Mine was actually working for a while and I was able to get into the bios, but I made some changes I cant remember and ever since then I have had the q code 6C error and I have no idea how to fix it. I tried reseting everything but no dice.
I am thinking if I updated the bios it should start working properly again but I don't know how if I can't even get into the bios to update it.








Any help would be great


----------



## TheBlademaster01

Have you tried removing some ram sticks?

E: I found this

http://www.overclock.net/t/1301388/asus-p9x79-pro-and-intel-i7-3820-no-post/0_30


----------



## Saieash

Quote:


> Originally Posted by *TheBlademaster01*
> 
> Have you tried removing some ram sticks?
> 
> E: I found this
> 
> http://www.overclock.net/t/1301388/asus-p9x79-pro-and-intel-i7-3820-no-post/0_30


I have removed ram sticks so I only had 1 stick per CPU and when that didn't make any difference, I used the ram working in my old pc (Corsair Vengeance 1600mhz) - and no dice.

My issue is that it was working for a while with no issue. I made a few bios changes that I can't remember and ever since then it stopped working. It wasn't immediately the *6C* q-coed error, there were a few other ones after I made those bios changes - I could see there was keyboard response (I could restart the PC using keyboard commands and the numberlock led would come on and off when it was pressed). I restarted the computer a few times and eventually landed at this Q code. Now with this q-code, there is no keyboard response at all - can't restart and no led response using the keyboard.


----------



## TheBlademaster01

6x codes are typically memory initialization errors. You could try to contact Asus.


----------



## ocvn

Quote:


> Originally Posted by *Saieash*
> 
> I have removed ram sticks so I only had 1 stick per CPU and when that didn't make any difference, I used the ram working in my old pc (Corsair Vengeance 1600mhz) - and no dice.
> 
> My issue is that it was working for a while with no issue. I made a few bios changes that I can't remember and ever since then it stopped working. It wasn't immediately the *6C* q-coed error, there were a few other ones after I made those bios changes - I could see there was keyboard response (I could restart the PC using keyboard commands and the numberlock led would come on and off when it was pressed). I restarted the computer a few times and eventually landed at this Q code. Now with this q-code, there is no keyboard response at all - can't restart and now led response.


Had same problem with newest bios last week. 6C, no display, I fixed it 2 times:
1st time, swap CPU which could helped me boot to bios but i didnt re-flash bios which make 6C code appear again after heavy heaven benchmark test.
2nd time, remove all PSU connection plus remove battery and clear cmos like 1 hours. After that, it booted to bios again and I flash the bios. Until Monday until now, it work rock solid.

Sorry for my bad E


----------



## TheBlademaster01

Quote:


> Originally Posted by *ocvn*
> 
> Had same problem with newest bios last week. 6C, no display, I fixed it 2 times:
> 1st time, swap CPU which could helped me boot to bios but i didnt re-flash bios which make 6C code appear again after heavy heaven benchmark test.
> 2nd time, remove all PSU connection plus remove battery and clear cmos like 1 hours. After that, it booted to bios again and I flash the bios. Until Monday until now, it work rock solid.
> 
> Sorry for my bad E


Very helpful post









Yes, swapping your CPUs resets the IMEI. 6x codes also seem to appear due to bad flashes (not sure what causes it though).


----------



## Saieash

Quote:


> Originally Posted by *TheBlademaster01*
> 
> 6x codes are typically memory initialization errors. You could try to contact Asus.


Quote:


> Originally Posted by *TheBlademaster01*
> 
> Very helpful post
> 
> 
> 
> 
> 
> 
> 
> 
> 
> Yes, swapping your CPUs resets the IMEI. 6x codes also seem to appear due to bad flashes (not sure what causes it though).


I haven't tried the CPU swap yet, but is there any one specific reason for the error? there seems to be several inconclusive explanations for this error on all the forums - not just this one. Could be for the CPU, RAM or Bios.


----------



## TheBlademaster01

BIOS flash and the IME (Intel Management Engine) most likely.


----------



## ocvn

Quote:


> Originally Posted by *TheBlademaster01*
> 
> Very helpful post
> 
> 
> 
> 
> 
> 
> 
> 
> 
> Yes, swapping your CPUs resets the IMEI. 6x codes also seem to appear due to bad flashes (not sure what causes it though).


With newest bios 5503, it could let me overclocking the ATI pci-e 3.0 card to 109 stable, however, c6 error come back. With bios 5303, solid rock bios but cant overclock pass 102 due to it doesnt have the option to choose the pci-e lane from 3.0 to 2.0. Hope asus know and release new bios which combine both bios above.


----------



## TheBlademaster01

How do you change the PCI-E settings?

I use 5503 though, but at stock. Which CPUs do you use.


----------



## ocvn

Quote:


> Originally Posted by *TheBlademaster01*
> 
> How do you change the PCI-E settings?
> 
> I use 5503 though, but at stock. Which CPUs do you use.


2697v2 bro. I am in 5303 atm. in 5503, check in the CPU sth and it about pci-e lane for all 7 pci-slot which let you configurate 1x-4x-8x-16 and pci-e gen1.1, gen2 and gen 3. Choose gen 2 for the vga card slot and you can boot at 109 np. 110 unstable.


----------



## TheBlademaster01

I don't fully understand what you are saying. But as I understand it there is a setting in BIOS that lets you pick PCI-E gen and bandwidth?

If you can write that in another language that would also be helpful. OC'ing is kind of a mystery on this board


----------



## ocvn

Quote:


> Originally Posted by *TheBlademaster01*
> 
> I don't fully understand what you are saying. But as I understand it there is a setting in BIOS that lets you pick PCI-E gen and bandwidth?
> 
> If you can write that in another language that would also be helpful. OC'ing is kind of a mystery on this board


you find this option yet? you run AMD or Nvidia card? I remember someone here run nvidia with 114, higher than AMD (my case is 109)


----------



## TheBlademaster01

I run an nVidia card. I have a 780 Ti at RMA currently and a GTX 470 as substitute


----------



## Stefan7

http://www.intel.com/content/www/us/en/quality/bga-socket-rework-video.html

some interesting watching about socket replacement - (if your a tech geek that is)

looking more like this isnt possible, i mean how many people would own a LGA2011 heat profile appart from the manufacturers.. could make one, and a jig, times and specific temperatures and the stencil for applying flux are a different matter lol


----------



## lowfat

Hopefully I will actually finish watercooling it this time.


----------



## Stefan7

http://www.okinternational.com/metcal/english/globalnavigation/products/convection-rework/system-series/nozzles/standard-nozzles found were to get the nozzles from, no idea of cost though, stencil kits on there too


----------



## Stefan7

Quote:


> Originally Posted by *lowfat*
> 
> Hopefully I will actually finish watercooling it this time.


Nice


----------



## eyepiz

Hello,

Until recently I have had my systems running rock solid!. nothing had changed until this Monday my workstation wouldn't boot it would go into attempting to repair on windows start up and no work, see the images below.

I first thought it was a bad hard drive so I connected another one, it wouldn't be recognized when attempting to load windows, I thought it might be a windows 8 thing so I tried windows 7 Pro same issues, cleared cmos reset everything I cant get a hard drive to work. I think it might be a UEFI issue but I don't know much about that. the motherboard recognized all the hard drives but I cant boot or load windows to any of them.

I would greatly appreciate any suggestions.

Thanks,








Workstation Specs

Asus Z9PE-D8 WS (Bios 5304)
(2) Intel Xeon 8 Core CPU's E5-2680v2 ES (QE5F)
(32) Gigs of G.Skill Trident X 2400 running @ 1600
(4) Evga Titan GTX in SLI
(1) OCZ Vertex 4 SSD (OS)
(1) Samsung Evo 840 (Data)
Enermax Max Revo 1500W PSU
Windows 8.1 Pro


----------



## halkx

It is the installation media that is not recognized...maybe you install from a USB stick connected to the usb3 ports of the motherboard?


----------



## eyepiz

I Tried both...Disk and USB3 Stick installs of Windows 8 Pro and Windows 7 Pro.


----------



## halkx

The exact same messages happened to me when I had the installation media hooked to a usb3 port...windows install didn't have the necessary drivers to recognize the ports so it popped these messages exactly...but if you have used a sata optical drive then you shouldn't have these messages...


----------



## TheBlademaster01

Yes, you need to install via USB 2.0 and one of the Intel SATA ports. I believe that you can't install to the Marvell ones (Windows that is, SUSE and Ubuntu install just fine either way).


----------



## squidman

I had a lot of install troubles trying to use USB install...it only worked for me using dvd-rom of win7x64. I seem to recall having usb devices plugged in gives the system problems on installing win7 or on boot...particularly with uefi bios..hope you work it out...







The install from usb never worked for me...







shame....am on .cap bios 3506 with dual qax8.....best of luck...but think it is having a usb device plugged in that's the problem...


----------



## samsue

Recently bought a Motherboards - Z9PE-D8 WS - ASUS and two Xeon 2670v2, I've read a lot about but have not got an objective and clear answer.

Supposedly the processdor shouldnt go to 3.3 Ghz However is limited to 2.5 even with Speedstep disabled and turboboos on or off

Clock Speed 2.5 GHz
Max Turbo Frequency 3.3 GHz

Can anyone tell me objectively what do i get to use all the power of the processor?


----------



## TheBlademaster01

All core turbo should be 2.9GHz. Do you have Engineering Samples? Also use HWInfo to monitor clock speeds. Enable speedstep and turbo.


----------



## samsue

No, and I use CPU-Z, other software causing blue screen problem (IRQL_NOT_LESS_OUR_MORE)

You're right, both activated and this now at 2.9 ghz ... It is not possible to reach the 3.3 Ghz? Specification Processor says 3.3 and not 2.9 ...

What puzzles me is that when I start Windows displays speed of 3.2 Ghz but when the system boots completely back to 2.5 to 2.9 Ghz ... like there's something limiting speed.


----------



## TheBlademaster01

3.3GHz (and even a tad higher) can be achieved if you load only 1 core. 3.2 if you load 2 cores, 3.1 if you load 3 cores, 3.0 if you load 4 cores. Anything more and you'll get 2.9GHz (as displayed in that chart). If you disable turbo, you only get 2.6GHz with 1 core and 2.5GHz if you load more.


----------



## samsue

Thank you very much! Solved my problem ... thanks for replying!

But I think this slutty, should be better specified in the Intel site.


----------



## TheBlademaster01

Yes I think so too







. It's really difficult to see which part is better that way. Just look at the chart above. 2680V2 and 2690V2 turbo equally hard on 1 core, but 2690V2 has a higher all core turbo. Even the numbers don't say much anymore since 2687WV2 isn't faster than 2680V2 at all...


----------



## MaDDeePee

what are these cheap chips from:

http://www.ebay.de/itm/XEON-E5-8-Core-1-6-2-3Ghz-2665-2650L-2660-2640-2630-2620-ES-LGA-2011-20Mb-/131172932418

They aren't named as "v2" but they're 22nm. Sandy-EP with 22nm? i am confused....
Also, 1.6Ghz that turbo (unknown cores) to 2.3Ghz.

Totally out of any spec.
Just downclocked 2640v2 ?


----------



## ccRicers

Quote:


> Originally Posted by *MaDDeePee*
> 
> what are these cheap chips from:
> 
> http://www.ebay.de/itm/XEON-E5-8-Core-1-6-2-3Ghz-2665-2650L-2660-2640-2630-2620-ES-LGA-2011-20Mb-/131172932418
> 
> They aren't named as "v2" but they're 22nm. Sandy-EP with 22nm? i am confused....
> Also, 1.6Ghz that turbo (unknown cores) to 2.3Ghz.
> 
> Totally out of any spec.
> Just downclocked 2640v2 ?


Yep. They're cheap because they're engineering samples. The "ES" gives it away. Sometimes ES chips are underclocked from the retail versions (and might have disabled other features, like hyperthreading).


----------



## mbraun

*Do you recommend this board for building 4-GPU systems ?*

Can it run 4 dual-slot GPUs at Gen 3.0 x16 PCIe? I read that sometimes Nvida switch to PCIe 2.0 with some chipsets or firmware that they consider unreliable. Some people earlier in this thread mentioned their PCIe bandwidth being lower than they expected. Is this what's happening here?

How is the reliability overall? Judging by the reviews on NewEgg and how often "BSOD" and posting problems are mentioned in this thread, I get the feeling it might not be too great?

BTW, I want to build a multi-GPU CUDA workstation (GTX 750 Ti and/or GTX 8xx, when they come out), running Linux. I haven't really made any other decisions, so any suggestions are very welcome.


----------



## TheBlademaster01

For PCI-E 3.0 across the board you need an Ivy Bridge Processor using slot 1, 3, 5 & 7 and no more.

Reliability as CUDA workstation should be fine if you get a working board (DoA rate is high) and non Sandy B stepping CPUs. I haven't gotten BSODs since using C0 chips.


----------



## mbraun

Thanks for replying!
Quote:


> Originally Posted by *TheBlademaster01*
> 
> For PCI-E 3.0 across the board you need an Ivy Bridge Processor using slot 1, 3, 5 & 7 and no more.


NVidia switches to PCIe 2.0 with Sandy Bridge?

Do I understand it correctly that the motherboard may come with a BIOS version that doesn't post with an Ivy Bridge CPU, and in this case, it would temporarily need a Sandy Bridge CPU to update the BIOS?
Quote:


> Reliability as CUDA workstation should be fine if you get a working board (DoA rate is high)


Like 5% or 50%?
Quote:


> and non Sandy B stepping CPUs. I haven't gotten BSODs since using C0 chips.


That's Sandy Bridge C0? (Wiki says the only Ivy Bridge C0 is a non-retail 12-core CPU used in a Chinese supercomputer)

I'm also thinking about going with SuperMicro, or maybe Tyan. When you want ECC RAM and 4+ dual-width GPUs, the choices are limited. Are the kinds of problems people have with this board likely to be there with SuperMicro and Tyan? And if not, why do people use it (at least those who are aware of the problems)?


----------



## Stefan7

Finally got the machine up and running and rendering albeit booting one in about 6-7 attempts.

When rendering it only used 7.5gb of ram out of the available 32gb. All 32 cores are maxing out though. So either thats all it needed or i need to change a setting somewhere, cant find any in revit. So possibly in the OS set up?...

Got the other broke board back resocketed!... Havent had the time to test it, however im not holding out much faith that it will work! the capacitors in the center are scorched and one is slightly bent. So I'm guessing the thermal tape didnt do the trick, guy said the temp reached 290 degrees!

Got a 460 gtx graphics card for £30 to down grade the quadro k4000 to try and sort out the booting issue as id rather sacrfice the Card than the CPU chips. Rvt doesnt utilise GPU rendering and thats what the workstation is for rendering in revit.

Pics of the resocket board

Removed socket


----------



## Stefan7

Is it possible to use an i7 processor in these boards i saw an ebay listing the other day saying that an i7 was for an LGA2011 socket, was this just an error in the listing?


----------



## Stefan7

Quote:


> Originally Posted by *TheBlademaster01*
> 
> 
> 
> All core turbo should be 2.9GHz. Do you have Engineering Samples? Also use HWInfo to monitor clock speeds. Enable speedstep and turbo.


wheres the e5-2665's?


----------



## TheBlademaster01

Quote:


> Originally Posted by *Stefan7*
> 
> Is it possible to use an i7 processor in these boards i saw an ebay listing the other day saying that an i7 was for an LGA2011 socket, was this just an error in the listing?


I think you could run them in the board if you only use 1 socket. Haven't seen any proof though. I do use 4-way chips in my motherboard though (E5-4650)
Quote:


> Originally Posted by *Stefan7*
> 
> wheres the e5-2665's?


That's a chart of only V2 chips.

I believe 2665 retail turbos up to 3.1 GHz on 1-2 cores, 3.0 GHz on 3-4 cores, 2.9 GHz on 5-6 cores and 2.8 GHz on 7-8 cores.

Can you post a pic of the scorched and bent capacitors?


----------



## Stefan7

Quote:


> Originally Posted by *TheBlademaster01*
> 
> I believe 2665 retail turbos up to 3.1 GHz on 1-2 cores, 3.0 GHz on 3-4 cores, 2.9 GHz on 5-6 cores and 2.8 GHz on 7-8 cores.


thats interesting. I'll take a close up pic tonight


----------



## TheBlademaster01

I made this yesterday



In general

1st bin = all core turbo

2nd bin = 1 core pair disabled

3rd bin = 2 core pairs disabled

4th bin = 3 core pairs disabled

There are some exclusions however

E5_2600.pdf 149k .pdf file


----------



## squidman

Nice graphs! Will find a use for the beast soon, my school is using older 2600k's for workstations and some dirt cheap graphics cards....not that it matters for basic modelling, but wouldn't want to do anything hefty! They ARE very quiet tho!

Can see a lot of teamviewer to home workstation in my future....


----------



## hemrie

Hey guys, I was wondering if I can get a little help.

I was trying to install 2 GTX Titans and could not get my computer to recognize more than one Titan at a time. I tried all combinations of slots.

I have one Titan black and one regular Titan.

My system specs are:
ASUS Z9PE-D8 WS
Dual -Xeon 2687w ES 3.0GHz QA91 (Bios - 3302)
Dual -CORSAIR Hydro Series H80i Water Cooler
EVGA GTX Titan Black and ASUS GTX Titan
Kingston 64GB (4 x 16GB) DDR3 1600Mhz SDRAM ECC Registered KVR16R11D4K4/64
SAMSUNG SSD 840 PRO
Hitachi 1.5TB
Western Digital Raptor 74Gb
PIONEER BD-RW BDR-209D
Rosewill Blackhawk Case

OS - WIN 8.1 PRO SP1 (x64) Build 9600

Is problem the old bios I have to use for the cpus?

I may be able to get either dual E5-2680s (QB79), dual E5-2690s (QB7S) or dual E5-2687w (QBE8) Would either of those allow me to run a later bios and improve support for my cards?

Also, how do I get the board to run the Titans at PCIe v3? HWiNFO64 shows the Titan Black running at PCIe v2.

Thanks in advance for any help that I may receive!


----------



## TheBlademaster01

Quote:


> Originally Posted by *hemrie*
> 
> Hey guys, I was wondering if I can get a little help.
> 
> I was trying to install 2 GTX Titans and could not get my computer to recognize more than one Titan at a time. I tried all combinations of slots.
> 
> I have one Titan black and one regular Titan.
> 
> My system specs are:
> ASUS Z9PE-D8 WS
> Dual -Xeon 2687w ES 3.0GHz QA91 (Bios - 3302)
> Dual -CORSAIR Hydro Series H80i Water Cooler
> EVGA GTX Titan Black and ASUS GTX Titan
> Kingston 64GB (4 x 16GB) DDR3 1600Mhz SDRAM ECC Registered KVR16R11D4K4/64
> SAMSUNG SSD 840 PRO
> Hitachi 1.5TB
> Western Digital Raptor 74Gb
> PIONEER BD-RW BDR-209D
> Rosewill Blackhawk Case
> 
> OS - WIN 8.1 PRO SP1 (x64) Build 9600
> 
> Is problem the old bios I have to use for the cpus?
> 
> I may be able to get either dual E5-2680s (QB79), dual E5-2690s (QB7S) or dual E5-2687w (QBE8) Would either of those allow me to run a later bios and improve support for my cards?
> 
> Also, how do I get the board to run the Titans at PCIe v3? HWiNFO64 shows the Titan Black running at PCIe v2.
> 
> Thanks in advance for any help that I may receive!


Yes, you would need one of those CPUs to get the board to recognize both Titans. The BIOS version is not the problem here, but the B0 chips are.

About PCI-E 3.0, I think you need Ivy Bridge CPUs for that. Do note that you need BIOS version 5206 to be able to POST with Ivy chips. I'm not sure about the last part, my E5-4650s runs my GTX 780 Ti at Gen 2 speeds.

@squidman

Thanks. Yes, I also have a 2600K system but it is really slow compared to the dual Xeons


----------



## hemrie

Thanks TheBlademaster01!

Would any S-Spec QB?? solve my problem for using multiple titans?


----------



## TheBlademaster01

Yes. QBxx are typically C0/C1 Sandy chips. They are as close to retail as you can get (called QS = Qualification Sample). The PCI-E 3.0 issue is an ES2 problem (Sandy B0).

Just to make sure it is not a faulty card, they do post on their own right? (only one at a time)

Also list the S-SPEC number of the chip you plan to buy so I can help double check compatibility


----------



## hemrie

Thanks for the assist TB01!

Here are the chips I am checking out::

dual e5-2690 - QB7S
dual e5-2680 - QB79
single e5-2689 - QB7A

Both titans post when used on their own on a P9X79. But the non-black titan would not post at all on the Z9PE-D8 WS. I returned that one.


----------



## TheBlademaster01

Yes, those should all work. I've got the 4-way version of the QB79. Dual QB7S is the fastest though out of those.

The fastest Sandy chip is QB7R.


----------



## hemrie

Cool.

I'll post back when I get the chips.

What BIOS version are you running with those QB79(s)?


----------



## TheBlademaster01

5503

I have QB83 though, not QB79. They are similar, but have 4 QPI links rather than 2.


----------



## sand74

@ Stefan7

Hi,

Did you self resocket?

I have a problem with a 2011 socket too.

I buy a new socket.
I`d like resocket the old one and socket the new one. You discribe 290°C?

Can you discribe your procedure?


----------



## sunny7day

can i get two e5 1660 running on this board?


----------



## ZDngrfld

Quote:


> Originally Posted by *sunny7day*
> 
> can i get two e5 1660 running on this board?


No. The first digit after E5 indicates how many possible CPUs you could run. E5-16xx is one, E5-26xx is two, E5-46xx is four.


----------



## Jann

Has anyone here actually tried using more than 4 GPU's with this board? For rendering/folding/mining/whatever loads them full.


----------



## ocvn

Quote:


> Originally Posted by *Jann*
> 
> Has anyone here actually tried using more than 4 GPU's with this board? For rendering/folding/mining/whatever loads them full.


Yes, i use it for 4 x 290x for mining before, just upgraded to 2 x 295x2


----------



## Stefan7

Slightly scorced and bent capacitor, could still be ok though


A tight fit in the small form factor case!


Shiney lights!


boots up everytime now with the 465gtx









what would be the cheapest chips i could buy to test the other motherboard i have? dont want to disassemble this other one + fry some expensive chips if every thing goes horribly wrong


----------



## Comex

Hallo,

a few days ago i have build my new workstation, but unfortunately i have troubles to boot the system.

After i assembled the system, i try to start it - no Post, even Q-Code don't light up.

So i have read in this forum about incompatibilities between E5-2600v2 and Rev. 1.04, after that i have order from Biosflash new BIOS-Chip (v. 5503)

Today i have replace old bios-chip with the new one, but the results are the same - no Post and Q-Code is still blank.
I also try to remove all RAM-DIMMs, - same result, even with single CPU - no Post and no Q-Code.

If some of you have any idea how can i fix it, i will be glad to hear it !


----------



## TheBlademaster01

I think you might have a DoA board. Have you tried booting the system outside of the case (on a piece of cardboard for example)

I first ran my system like this


Spoiler: Warning: Spoiler!


----------



## Comex

Thanks a lot *TheBlademaster01* !!!!!!

You just made my day !!

Now everything work just fine !

Thanks again !!!


----------



## TheBlademaster01

No problem, glad it worked for you









With complex systems like these it's always good to test as little as possible at a time. There are just a lot of things that can go wrong. I take it that you just had a short inside your chassis (maybe a standoff that didn't line up correctly?)


----------



## Comex

I guess one of the reasons was, that I've tied all screws too much and another one maybe because the case it self ( Corsair Graphite 760T) support *E-ATX* and not *SSI-EEB*. So not all of the screw holes matched with those from MB.


----------



## Reptarr

Hello Everyone,

I have an ASUS Z9PE-D8 WS motherboard with dual Xeon 2620 CPUs, dual Quadro K6000, and one GTX Titan Black. I have populated the GPUs in the following PCIe arrangement:

PCIe slot 1 - Quadro K6000
PCIe slot 3 - Quadro K6000
PCIe slot 5 - GTX Titan Black

All the other PCIe slots are empty. The specifications for the expansion slots of this motherboard are:

PCIE 1,3 - CPU 1,x16
PCIE 5,7 - CPU 2,x16
PCIE 6 - CPU 2,x8
PCIE 2,4 - CPU 2,x8

Therefore according to the specs, all three cards should be running in x16 mode. However, there are two problems that I spotted from GPU-Z.

1.) One of the Quadro K6000's is running in x8 mode.
2.) The GTX Titan Black is running in PCIE 2.0 configuration, while both Quadros are running in PCIe 3.0 configuration.

Can anyone tell me what might be happening, or a way I can debug these two issues?

Thank you for your help.


----------



## happy hopping

Quadro uses totally different driver than GTX. What if you pull those 2 quadro and put any Nvidia card in, can you duplicate the problem?


----------



## Reptarr

Do you mean any other NVIDIA GTX cards? I can give that a shot and see. Thanks for the suggestion!


----------



## Reptarr

I removed the two Quadros, and replaced them with a GTX Titan Black (slot 1) and GTX Titan (slot 3). All other slots are empty. Replacing them directly didn't solve these issues. However, I solved issue #2 by creating an RMPcieLinkSpeed registry in regedit.

Unfortunately issue #1 is still there, GTX Titan is running at x8 while GTX Titan Black is running at x16. Does anyone have any suggestions as to why this is?

Thank you for your help.


----------



## happy hopping

I have seen LCD problem w/ nvidia driver, only fixed months later. Best guess is, this is a bug in the driver


----------



## BrotherBeast




----------



## TheBlademaster01

Great find









Where did you find it?


----------



## BrotherBeast

I was googling for it and a page came up linked to the D16 model on asus.com.It was in the banner for their workstation motherboard section for there commercial products.Links to product page was missing or broken.Just the image.


----------



## halkx

http://www.asus.com/Commercial_Servers_Workstations/Z10PED8_WS/

And here are the relevant info from asus' site...


----------



## halkx

Intel Xeon E5-2699 v3 (2.3G,145W,L3:45M,18C,HT)... Huh?

I can only imagine what the price of this would be...lol

Retail little under 4k euros...lol


----------



## TheBlademaster01

$4k


----------



## MaDDeePee

I know i cant plug a V3 CPU on the Z9PE but would it be possible to plug V2 CPUs on this V3 socket?

So...is it backward compatible or are that sockets physically incompatible?


----------



## TheBlademaster01

Physically incompatible. They have different ILM keys and different pin layout.


----------



## halkx

Question...I acquired 2 asus gtx 780 dcuii oc cards for very low price. What slots should I use to have optimal results?


----------



## TheBlademaster01

Any of the blue ones and depending on your applications either the first two or the last two.


----------



## halkx

What's the difference between the top two and the bottom two slots?


----------



## TheBlademaster01

The top two run from CPU 0 and the last two run from CPU 1. It's best to have both GPUs connected to the same host because of latency. I know that Radeon R9 290 series even stutter when they aren't connected to the same host in games due to the fact that AMD runs crossfire via the DMA engine.


----------



## phillydee

Hey Blademaster or anyone with more expertise than me with these boards + GTX cards:

I'm thinking of upgrading to a GTX980... do you think it'll play nicely with the mobo? Not planning any SLI for now... thanks


----------



## TheBlademaster01

You should have no problems if you use C0 or later Sandy or Ivy chips and BIOS v3302 or later


----------



## phillydee

Quote:


> Originally Posted by *TheBlademaster01*
> 
> You should have no problems if you use C0 or later Sandy or Ivy chips and BIOS v3302 or later


OK thanks, I double checked and I do have rev.C0 chips. I'm on Bios 5-something now so I think I'm in the clear.

+Rep'd. Thanks!


----------



## TheBlademaster01

What card are you running now?


----------



## phillydee

a GTX680.


----------



## TheBlademaster01

Okay, if you didn't do so already. You can install the PCI-E 3.0 patch to enable GEN 3 PCI-E on your system

http://www.techpowerup.com/downloads/2148/nvidia-geforce-kepler-pcie-3-0-mode-enabling-patch-for-sandy-bridge-e-systems/mirrors


----------



## halkx

PCI-E 3.0 patch? Where do I find it? Will I need it when my 780s arrive?


----------



## halkx

Ah...ok...lol


----------



## dec0da

hello,

i built a zpe9-d8-ws rig recently, for sw dev purposes. I started with one E5-2640v2 cpu and 64 gb registered ECC Ram. everything went flawlessly, the system booted at first power-on. i use windows 8.1 pro, with a bunch of dev environments virtualized with vmware workstation.

i use an old zotac gt 630 card (1 or 2gb video ram), which allows two drive two full hd monitors.

then i decided to add a second cpu and another 64gb ram, again the system boots and works immediately after putting in the additional components. however, windows now fails to detect the graphics card and switched back to the default vga drivers.

(i use latest zpe9-d8-ws bios, latest nvidia drivers etc.)

i tried all pci ports, no difference. the funny thing is: if i remove one of the ram modules (thus 112gb instead of 128 gb of ram), the graphic card works just fine again.

to me, this sounds like some address space conflict/limitation? so may i ask one of the users who are experieced with those setups with high memory volumes, is there a specific bios setting i am missing? what (low end) graphics adapter would you recommend, that are known to work? or is a known limitation with this board?

regards,
rene


----------



## halkx

@blademaster

my 780s arrived...installed and running fine...I dled the kepler patch for pcie 3.0 and tried to install but gpuz still reports the link as 2.0...do I need the patch or not?

thanks


----------



## TheBlademaster01

Quote:


> Originally Posted by *dec0da*
> 
> hello,
> 
> i built a zpe9-d8-ws rig recently, for sw dev purposes. I started with one E5-2640v2 cpu and 64 gb registered ECC Ram. everything went flawlessly, the system booted at first power-on. i use windows 8.1 pro, with a bunch of dev environments virtualized with vmware workstation.
> 
> i use an old zotac gt 630 card (1 or 2gb video ram), which allows two drive two full hd monitors.
> 
> then i decided to add a second cpu and another 64gb ram, again the system boots and works immediately after putting in the additional components. however, windows now fails to detect the graphics card and switched back to the default vga drivers.
> 
> (i use latest zpe9-d8-ws bios, latest nvidia drivers etc.)
> 
> i tried all pci ports, no difference. the funny thing is: if i remove one of the ram modules (thus 112gb instead of 128 gb of ram), the graphic card works just fine again.
> 
> to me, this sounds like some address space conflict/limitation? so may i ask one of the users who are experieced with those setups with high memory volumes, is there a specific bios setting i am missing? what (low end) graphics adapter would you recommend, that are known to work? or is a known limitation with this board?
> 
> regards,
> rene


Never seen anyone with the same problem. What BIOS version are you running? And which OS + version?

Quote:


> Originally Posted by *halkx*
> 
> @blademaster
> 
> my 780s arrived...installed and running fine...I dled the kepler patch for pcie 3.0 and tried to install but gpuz still reports the link as 2.0...do I need the patch or not?
> 
> thanks


Did you already force Gen 3 via BIOS? If you already applied the patch, it should work after doing this.


----------



## dec0da

thx TheBlademaster01 for reply!
Quote:


> Never seen anyone with the same problem. What BIOS version are you running? And which OS + version?


-BIOS version 5601 from 2014/09/04 (same with preceding version)
-Windows 8.1 Professional (512GB limitation)
-nvidia driver 344.11 WHQL from 2014/09/18 (same with preceding version)

I will get another zotac gt 640 2gb the next couple of days for tryout, will report if the issue remains.

regards,
rene


----------



## TheBlademaster01

Thanks, please do. You might help someone else with the same issue.

I don't think you should be at a limit with those components and this board. It does seem like that however judging by the symptoms.


----------



## Jeffinslaw

Hey guys, if there is anyone lurking in this thread that are interested in purchasing one of these boards, I have one up for sale in the marketplace! Go take a look


----------



## dec0da

hello,

regarding the problem with the inability to use all memory slots when using the zotac gt630 1gb video card, i tested it now with another zotac gt640 2gb card, and the problem diappeared.

maybe this info helps someone in the future..


----------



## paxilon

I'm having trouble with the GTX 980 on this motherboard, I can't seem to get anything on the display. I have Bios 5-something and an Intel Xeon E5-2687W ES 3.00 GHz Processor. The build works with the quadro 4800 video card, but not the GTX 980. Any suggestions as to how to fix this?


----------



## wirk

Quote:


> Originally Posted by *paxilon*
> 
> I'm having trouble with the GTX 980 on this motherboard, I can't seem to get anything on the display. I have Bios 5-something and an Intel Xeon E5-2687W ES 3.00 GHz Processor. The build works with the quadro 4800 video card, but not the GTX 980. Any suggestions as to how to fix this?


If quadro is OK and 980 not it sounds driver problem probably not tested with this mobo, report this to Raja from Asus on one of the official support forums.


----------



## ZDngrfld

Quote:


> Originally Posted by *paxilon*
> 
> I'm having trouble with the GTX 980 on this motherboard, I can't seem to get anything on the display. I have Bios 5-something and an Intel Xeon E5-2687W ES 3.00 GHz Processor. The build works with the quadro 4800 video card, but not the GTX 980. Any suggestions as to how to fix this?


It sounds like you're running B0 or B1 CPUs... They don't like newer video cards.


----------



## TheBlademaster01

Quote:


> Originally Posted by *dec0da*
> 
> hello,
> 
> regarding the problem with the inability to use all memory slots when using the zotac gt630 1gb video card, i tested it now with another zotac gt640 2gb card, and the problem diappeared.
> 
> maybe this info helps someone in the future..


Thanks for reporting back. It's probably the BIOS of the old card. I don't think I've ever seen anything like it tbh.
Quote:


> Originally Posted by *paxilon*
> 
> I'm having trouble with the GTX 980 on this motherboard, I can't seem to get anything on the display. I have Bios 5-something and an Intel Xeon E5-2687W ES 3.00 GHz Processor. The build works with the quadro 4800 video card, but not the GTX 980. Any suggestions as to how to fix this?


You would need to use the Quadro, or get new CPUs and use the GTX 980. New CPUs meaning C0 are later Sandy chips or Ivy chips. Make sure you are not on BIOS 5004 if you decide to go with Ivy.

Quote:


> Originally Posted by *ZDngrfld*
> 
> It sounds like you're running B0 or B1 CPUs... They don't like newer video cards.


Yeah it sounds like B0 (QA90/QA91)


----------



## ocvn

Anyone using raid card? Currently i installed LSI raid card, first boot ok but after hours, the mobo didnt want to boot to bios anymore and usually show code: 32 and reboot loop. Anyone havr the problem with addon card? I also tried with mushkin scorpion pci-e ssd and same issued. Tried all bios from 5301 to 5601, all 3 slots:5-6-7. Cant install from 1-4 because install AMD VGA already.


----------



## knopflerbruce

I grabbed an Ivy-E 2696 (?) B0 ES off ebay yesterday. Was sort of cheap, so I thought 'why not?'. But, as it is an older stepping I'm not sure what board I should get... are THESE B0's fine on the Z9PE? I've read a few pages, but most of the B0 talk seems to be about SB-E B0's, and this is definately not one of those.

I'm even thinking of testing it in a desktop mobo, just for fun - but I only have a Gigabyte X79-UD5 that I can use.


----------



## dec0da

Quote:


> Anyone using raid card? Currently i installed LSI raid card, first boot ok but after hours, the mobo didnt want to boot to bios anymore and usually show code: 32 and reboot loop. Anyone havr the problem with addon card? I also tried with mushkin scorpion pci-e ssd and same issued. Tried all bios from 5301 to 5601, all 3 slots:5-6-7. Cant install from 1-4 because install AMD VGA already.


i use a LSI SAS 9207-8I SGL HBA since a week now wihtout issues, pci slot #3.
maybe its an issue with temperature, since those cards get quite hot but are shipped without a fan.
does the problem remains when removing (some of the) graphic adapters temporarily?


----------



## ocvn

Quote:


> Originally Posted by *dec0da*
> 
> i use a LSI SAS 9207-8I SGL HBA since a week now wihtout issues, pci slot #3.
> maybe its an issue with temperature, since those cards get quite hot but are shipped without a fan.
> does the problem remains when removing (some of the) graphic adapters temporarily?


Yes, tried remove vga already but the issued remain. Qcode 32. Took the card out, work smooth and stable. Cant install to another slot due to 2 amd card must connect to same cpu... Today i tried with my asus impact and also the bootloop.. Might be faulty card??? Anyone have experience with lsi raid?


----------



## poulk

Hello all!
I forget show here my own build on this asus mb make at begin of 2014








2x 12core xeon v2(2,5 Ghz oc)/128GB ram/GTX Titan/ LSI megaraid(8xhitachi 2tb RE) raid 0(IBM oem)/ ssd 512gb plextor m5pro/psu sf 1300w/full watercooled


----------



## squidman

Quote:


> Originally Posted by *TheBlademaster01*
> 
> Okay, if you didn't do so already. You can install the PCI-E 3.0 patch to enable GEN 3 PCI-E on your system
> 
> http://www.techpowerup.com/downloads/2148/nvidia-geforce-kepler-pcie-3-0-mode-enabling-patch-for-sandy-bridge-e-systems/mirrors


Wondering whether this patch will solve my problem...on 'old' QA8X B0 e5-2665's, and the mobo just doesn't like the blue pci-e slots...just have the one gtx 770 /aka gtx 680. Am on a .cap bios, the last bios that supports B0 sandy chips...(.5xxx? forget long night of maya modelling here...)..

the bummer for those of us with the z9pe is that taobao prices for ES v3 chips are in fact cheaper than the old v2 chips (such as e5-2697 v2)...in fact seen 14-core v3 chips for cheaper than aforementioned...




























... of course need to factor in price of z10pe mobo and new ram....

Edit: how on earth do you upgrade to a new mobo when you have a raid 5 system? Just the thought terrifies me...









Wow great to see the old smileys! New ones suck...


----------



## birkett

Hopefully someone here can help me out with this.

I own a Z9PE-D16 (the server oriented version of the -D8 WS). I've been running a Radeon 5770 with it for a while now, and am looking to upgrade.
Have the graphics card issues been largely resolved? I am looking at a GTX970, and have C2 stepping E5-2620v1's. Any problems to be expected with doing this?


----------



## Stefan7

well more problems with the board today surprise surprise. In the middle of scanning a document last night it shut its self down. Now refuses to post at all, no idea what the hell is wrong with it. Its been working fine for about a month rendering all sorts. Have had some issues with windows starting up and not been able to repair the windows start up part. I move the the graphics card into pci slot 2 and it posted. Got to disk windows disk check 0% then crashed. I have removed the battery removed all power connects and reconnected. tried booting without drives conected or powered. Nothing. Absolutely had enough. Feed up is an understatement. What is concerning me is that it doesnt even display numbers for fault codes at all. so looks like maybe a power surge happened last night. but y did it post once when i moved the graphics card?/?? confused


----------



## Stefan7

alright getting somewhere, removed the CMOS Jumper under the BIOS Chip and put it back, now getting Q-Code 62 ?


----------



## birkett

62 is SouthBridge initialization. So it's likely waiting for the graphics card PCI-E link to train. Have you got another card to test in there?


----------



## TheBlademaster01

Quote:


> Originally Posted by *Stefan7*
> 
> alright getting somewhere, removed the CMOS Jumper under the BIOS Chip and put it back, now getting Q-Code 62 ?


Yeah, it is actually b2 which means it is stuck in OpROM initialization (the process where it probes for I/O devices w/ a BIOS -- like your GPU). This happens a lot with Sandy B0 and .CAP BIOSes especially when the card is in a slot other than #3. Even in #3 it will only POST like 1 out of 3 times.


----------



## ocvn

for your info, with the PCI-E problem, you can switch the tesla jumper to non-DF position (PCH position to BMC). Before I can't installed both of my PCI-E SSD and raid (qcode 62, 32), after switching it, It work flawlessly.


----------



## Stefan7

yeah, i didnt plug the 2x 6pin power supplies into the graphics card! what a dope ha. Its back up and running now albeit out of case, think there may have been a cable touching or something maybe. I have Q-Code 15 now still.

My next question is its always saying on the trouble shooting that the driver for the standard VGA Graphics adapter isn't installed.

What is that?.. says its in the pci-e slot 8 but theres only 7 lanes?...i cant see a vga plug anywhere haha.
Guessing the driver can be found on Asus website?


----------



## Stefan7

also what would be the best graphics card to run with the these Sandy bridge B0 Stepping chips and early bios?

the bios im running is 0503

So my errors in windows trouble shooter are -

Standard VGA driver not installed
SAS Controller driver not installed
Marvell Console ATA driver not installed

I have the drivers but cant seem to install them.

Im wondering if these devices should be disabled in bios so windows doesn't see them as I don't think im using these ?

thanks for the advise folks


----------



## ocvn

Quote:


> Originally Posted by *Stefan7*
> 
> also what would be the best graphics card to run with the these Sandy bridge B0 Stepping chips and early bios?
> 
> the bios im running is 0503
> 
> So my errors in windows trouble shooter are -
> 
> Standard VGA driver not installed
> SAS Controller driver not installed
> Marvell Console ATA driver not installed
> 
> I have the drivers but cant seem to install them.
> 
> Im wondering if these devices should be disabled in bios so windows doesn't see them as I don't think im using these ?
> 
> thanks for the advise folks


Intergrated vga. Check manual. They have the jumper for disable it.


----------



## Franz Obermeyer

http://www.overclock.net/t/1526400/asus-z9pe-d16-2l-2nd-cpu-not-recognized-is-it-the-memory

I started this thread on problems with memory on my asus motherboard before I found this thread. Better I think to not clutter this thread with just my problem. If knowledgeable people could go there & read the thread, I'd appreciate it. I'll read this thread in it's entirety now, on the chance there's something useful. Thanks in advance.


----------



## ocvn

Can we call WR for 2697v2


----------



## TheBlademaster01

Wow, that is one sick OC man







. How did you get it to clock that high? Can you run CineBench at that speed? My E5-4650s clock like dogs...


----------



## ocvn

Quote:


> Originally Posted by *TheBlademaster01*
> 
> Wow, that is one sick OC man
> 
> 
> 
> 
> 
> 
> 
> . How did you get it to clock that high? Can you run CineBench at that speed? My E5-4650s clock like dogs...


The key for OC with this board is CPI-E and power consumption I didn't take the screenshot of cinebench but lin-x 10 hours is can call stable clock right Note that the system run with dual R9-295x2 so i dont know with other graphic cards i can push it to this limit or not (115 is no post but if you have e5-2687v2 with 114 bclk, i think that's will be benchmark monster )


----------



## Stefan7

How she currently sits after tripping out the other day, now have a proper surge protector ! back running apart from graphics card kernel error message ever two minutes

Really happy with the be quiet stuff btw pretty much silent with all 8 fans and PSU

Thanks for the advise will look at the instruction book about standard VGA its numbered on a diagram just haven't got time to read yet as too much work to do.

Still none the wiser as to what graphics card to buy that will be stable with bios and chips.

was thinking about trying out a Quadro FX3800 see how that goes. this GTX 465 is abit pants it has an after market Gelid cooler and bits of heatsink stuck on with double sided tape









I know I couldn't flash to a stable bios with the quadro K4000 card however do you think that tesler jumper might help is that what your saying?

Also can older chips like B0 E5-2665s be over clocked?

amateur computer bodger that I am


----------



## ocvn

Quote:


> Originally Posted by *Stefan7*
> 
> 
> 
> How she currently sits after tripping out the other day, now have a proper surge protector ! back running apart from graphics card kernel error message ever two minutes
> 
> Really happy with the be quiet stuff btw pretty much silent with all 8 fans and PSU
> 
> Thanks for the advise will look at the instruction book about standard VGA its numbered on a diagram just haven't got time to read yet as too much work to do.
> 
> Still none the wiser as to what graphics card to buy that will be stable with bios and chips.
> 
> was thinking about trying out a Quadro FX3800 see how that goes. this GTX 465 is abit pants it has an after market Gelid cooler and bits of heatsink stuck on with double sided tape
> 
> 
> 
> 
> 
> 
> 
> 
> 
> I know I couldn't flash to a stable bios with the quadro K4000 card however do you think that tesler jumper might help is that what your saying?
> 
> Also can older chips like B0 E5-2665s be over clocked?
> 
> amateur computer bodger that I am


Tesla jumper which let the pci-e compatible with quite alot of device. I dont know othercase but with my muskin pci-e, LSI raid and soundcard, all of them work when i switched tesla jumper.
I dont have sandy-ep to test but with ivy, from before, my most stable was 106, now i can move to 114 (115 is pci-e limit).


----------



## woops

Hello,

Can you please tell me the driver file name needed for "SAS Controller"?

Also, is it possible to boot using usb 3.0 instead of the usb 2.0?

Thank You.


----------



## Stefan7

I havent had chance to find out about the SAS Controller and the Marvell SATA console yet.

However moving the Jumper to disable the standard onboard VGA has solved my graphics card driver issues so happy days!!









Oh tried moving the teslar jumper to test out if this would make the Quadro K4000 Work...No luck it didnt post
so still using the gtx465


----------



## woops

I need to clarify: the case i am using (define xl r2) has front case USB 2.0 x 2 and USB 3.0 x 2. You have to plug the one connector for all these usb ports into USB3_34 of the motherboard. The problem is when i want to boot from a usb flash drive by holding down F8, the BIOS will recognize the 2.0 but not the 3.0.

Is there a way to fix this in order to boot from 3.0 usb port?

Thanks for some ideas or solutions.


----------



## geort45

HALP!

I just bought an used Z9PE, couldn't resist! I'm supposedly covered by ebay but I'd prefer if it worked haha. Point is, it has some (3-4) slightly bent pins, I can straighten them up with a magnifying glass but I have to test it! Point is... I haven't bought the Xeons yet! I only have some i7's. I KNOW this mobo wouldn't work with 2 i7's, but I just wanna test it with 1 of them! Would it work? At least my i7 won't be fried?

I have a 3770K and a 2600K btw

THANK YOU!


----------



## lowfat

Neither of those cpus will physically fit in the socket.


----------



## geort45

Quote:


> Originally Posted by *lowfat*
> 
> Neither of those cpus will physically fit in the socket.










am I ******ed or what! I totally oversaw that! I forgot there were 1155's and 2011's DAMN!


----------



## aerosmith9110

Hi guys/gals,

I'm planning to purchase this board for my dual e5-4650.

1. Want to know what bios i need.
2. Is this ram compatible:
Product Name 8GB 2RX4 PC3L-10600R-9 Kit
Product Type RAM Module
Memory Standard DDR3-1333/PC3-10600
Memory Technology DDR3 SDRAM
Memory Size 8 GB
Memory Speed 1333 MHz
Compatibility HP ProLiant BL460c G6 Blade Server
Number of Modules 1 x 8 GB
3. Quad hd7950 won't have a problem?


----------



## TheBlademaster01

Quote:


> Originally Posted by *woops*
> 
> I need to clarify: the case i am using (define xl r2) has front case USB 2.0 x 2 and USB 3.0 x 2. You have to plug the one connector for all these usb ports into USB3_34 of the motherboard. The problem is when i want to boot from a usb flash drive by holding down F8, the BIOS will recognize the 2.0 but not the 3.0.
> 
> Is there a way to fix this in order to boot from 3.0 usb port?
> 
> Thanks for some ideas or solutions.


I don't think it's possible. Windows also doesn't want to install to drives connected to the Marvel SATA 6G ports. I think it's the same for the USB 3.0 ports since they are run off the ASmedia controller. I'm not 100% sure. I just know it doesn't work for me.
Quote:


> Originally Posted by *aerosmith9110*
> 
> Hi guys/gals,
> 
> I'm planning to purchase this board for my dual e5-4650.
> 
> 1. Want to know what bios i need.
> 2. Is this ram compatible:
> Product Name 8GB 2RX4 PC3L-10600R-9 Kit
> Product Type RAM Module
> Memory Standard DDR3-1333/PC3-10600
> Memory Technology DDR3 SDRAM
> Memory Size 8 GB
> Memory Speed 1333 MHz
> Compatibility HP ProLiant BL460c G6 Blade Server
> Number of Modules 1 x 8 GB
> 3. Quad hd7950 won't have a problem?


I assume those are retail 4650s? It doesn't seem like this will be an issue, but you'd have to ask Asus to make sure. Any BIOS will work with retail 4650s.

I'm running 5206 on my system (also dual E5-4650). You do need 3109 or later if you want to run Win 8 or later. Also, 3302 or later for PCI-E 3.0 GPUs. I'd upgrade to 5206 or just the latest. I haven't tested it since I have no problems with 5206.


----------



## aerosmith9110

Thanks TheBlademaster01.

This is Revision C2 Got it from IBM servers ( Retail?).

So Just to be sure I got it correct, It is unlikely for the memory to have a problem? Its going to work? Asus Tech Support Won't be able to help out as Ram isn't one of those famous brands ( Ramaxel ).


----------



## TheBlademaster01

It's unlikely to cause a problem. It may since we have no data on it, but I think it would work. And yes, those are retail with VT-d fix


----------



## aerosmith9110

GREAAT! I'm ordering the board at amazon later. it should arrive by 2nd week of next year at my country.

BTW, were you able to OC the chips? I am currently using the msi bigbang xpowerii and what ever I do the clock in cpu-z don't get past 2.9GHZ! even i set multiplier to 33. or increase the clock to a minimum of 105.


----------



## TheBlademaster01

I can get bClk stable to about 105MHz. It turbos to 31x 105MHz on all cores, so something like 3.26GHz. Since I have C0 E5-4650s they turbo 2 bins higher than retail E5-4650s on single core, so 35x 105MHz = 3.68GHz.

Do you have speedstep/C1E and turbo boost enabled?


----------



## aerosmith9110

Quote:


> Originally Posted by *TheBlademaster01*
> 
> Do you have speedstep/C1E and turbo boost enabled?


Yes, Ho w about your voltages? and will the turbo speed show on CPU z?

And I do apologize why is C0 better OC than C2? isn't C2 newer?

Do you know any reliable site I could read regarding OCing these?


----------



## TheBlademaster01

C0 aren't better. They are just clocked the same as an E5-2680 but with 4 QPI Links, and 2680s clock higher on 1-4 cores. They also don't have VT-d. C2 is newer and has fixed VT-d, but Intel clocked retail E5-4650s lower (max multi of 33x instead of 35x).

I need 1.025-1.050V System Agent voltage. Stock vcore.

They do show turbo speeds in CPU-Z, but I mostly use HWInfo64 in Windows.

I'm not sure where you can read up on OC'ing these. I mostly listened to ocvn and tested settings out for myself. Did you look at clock speeds while running a bench like CineBench or some other load?


----------



## aerosmith9110

Yeah I was looking at the speed while running p95. its up to 2.9ghz only.


----------



## TheBlademaster01

That really doesn't look correct. In fact, when you run the CPU out of the box, it should do at least 3.1GHz. It's just that I don't have experience with the board







...


----------



## aerosmith9110

ok got yah. will check the settings. tnx


----------



## RenoLevi

Hi

Another user with this board, Im having a serious problem and was wondering is anyone here could help me

i get crashes all the time mostly with using 3D softwares or games

I have a workstation based pc

Board - Asus z9-PED8-WS

Memory - Corsair XMS3 48gb

Gigabyte GTX 780 - (today i about ANOTHER 780 cause i couldnt figure out what the problem is, thought it was my card, but it wasnt, it happens again with the new 780)

PSU - 1200w TT

Things i tried in that order -

Running windows 8.1 64but

Latest Nvidia drivers,
Updated all my asus board drivers
Update bios on my mobo
Older Nvidia drivers - about 4 versions

Mem tests (no errors)

Clean install of win 7 pro 64bit on a new harddrive
Bought ANOTHER Gtx 780
Oldest Driver for 780

Cleaning dust, trying different PCIE slots...

nothing worked so far

im not sure whats next now

SysnativeFileCollectionApp.zip 755k .zip file


HealthReport.zip 173k .zip file

i added dmp files if needed


----------



## RenoLevi

Also, should i try disabling 1 CPU? (if its even possible from the bios)

Could it be CPU problem?


----------



## MaDDeePee

48Gb ram? Are you sure? How many slots populated?

Start with memtest86+ (google, download, run a whole night)


----------



## TheBlademaster01

Also, which CPUs do you use and what BIOS version?


----------



## RenoLevi

I have 32gb kit (4x8) + 4x4. All slots are full (though iv ALWAYS had problem that slot B2 shows 0mb no matter what stick i use, but pc worked fine for like a year)

Also tested with only the full 32 kit (4sticks)
And another test with only 1 8gb stick in A1

Got same error, same time

I have 2 cpus i running now, both E5 2680


----------



## TheBlademaster01

Those are retail?

In any case, try updating the motherboard BIOS to 5206.


----------



## RenoLevi

ES, but never had any problems
Bios update didnt help


----------



## TheBlademaster01

No, you don't need retail chips but certain ES chips may be buggy (B0, QAxx chips). Although seeing as your system POSTs just fine, I'm inclined to think something else is the problem.

What kind of driver errors do you get?


----------



## RenoLevi

its BBcode 116, something related to graphic card/link

the code indicated the display adapter tried to reset and basically "hung" or something like that

im not very good with debuging


----------



## RenoLevi

Latest attempt
Removing all devices leaving only 1 HDD SSD, disconnect LAN
installed OEM WINDOWS 7 64bit from original cd
Installed vga drivers from original Gtx Cd that came with the card
Installed 3Dmark
Fired up -> BLACK SCREEN AND REBOOT right away







((((


----------



## TheBlademaster01

Does the card work in a different PC?


----------



## RenoLevi

The guy i got it from said its works 100%

Is it possible to disable 1 cpu from the bios for a test?


----------



## RenoLevi

Tried my pc with only 1 cpu, (removed cpu no2)

Still black screen
Put it back on....

Recently it started to go black right on windows logon


----------



## TheBlademaster01

Have you tried using 1 CPU with 1 RAM stick or just a different GPU (something older)?


----------



## RenoLevi

CPU 1 with few sticks


----------



## RenoLevi

Also, this is my PSU, thermaltake thoughpower, 1200w, those are the ports

How should i connect my GPU? (It haz 1x6 pins and1x8pins)

Those 3v and 4v are confusing


----------



## RenoLevi

This is how i connected it

Red going to red 4v (6pin)
Black going to black (8pins) - no othe deviced on that cable

Should work fine right?


----------



## RenoLevi

Is it possible and perfectly fine to run this board with only 1 memory stick? Including games


----------



## TheBlademaster01

Running only one stick will of course perform really bad, but it is more to find out where the failure is.


----------



## RenoLevi

can it cause black screens/system crash? im trying to isulate the problem

i clueless now, really nothing else left to do other than to just replace the board

but im not sure its the motherboard,

i'd hate to go through all that to find out its not the board
(i have to order one oversea, its not available where i live)

also, i was never able to fix the 0mb issue on slot B1

any ideas about that?


----------



## RenoLevi

Today i got an old ATI CARD 4850, tested it, HAD NO ERRORS, NO CRASHES

what does it mean?


----------



## TheBlademaster01

780 doesn't work. So yeah, I'd say the Z9 is fine. Aside from the not functional RAM DIMM though.


----------



## RenoLevi

DISPLAY PORT ADAPTOR!!! I disconnected my apple 27 and no more crashes! Cant believe it! 2 weeks i changed hardware and reinstalled windows, it was the DP!!

Thanx for the help man


----------



## tomyboy

Quote:


> Originally Posted by *poulk*
> 
> Hello all!
> I forget show here my own build on this asus mb make at begin of 2014
> 
> 
> 
> 
> 
> 
> 
> 
> 2x 12core xeon v2(2,5 Ghz oc)/128GB ram/GTX Titan/ LSI megaraid(8xhitachi 2tb RE) raid 0(IBM oem)/ ssd 512gb plextor m5pro/psu sf 1300w/full watercooled


What a score ....


----------



## Stefan7

Finally ironed out all my issues with windows and board drivers.

Fixed graphics driver by moving the vga jumper.
Disabled marvell console in bios.

Fixed windows group policy login problem.

whoohoo. Merry Christmas z9ped-d8 owners!









Anyways...

I was looking at E5-2699 V3 chips today,

there is one on uk ebay for £1750

and some (24 pieces) on taobao for 10500 yuan so roughly £1100

Im guessing one would need to step it up an notch to the asus z10pe-d8 or maybe asus z10pe-d16?

Anyone any ideas?


----------



## halkx

Merry Christmas to you too...

Well, you definitely need to upgrade your board AND the memory because these chips don't work on Z9 boards...you need Z10 boards which require DDR4 memory sticks


----------



## Stefan7

I see thanks, they are out of my price bracket for now any way but i can







over them


----------



## RenoLevi

btw
whats the fastest/more powerfull xeon our board support today?

(i thought it can put any of the E5 series?)


----------



## halkx

http://www.asus.com/Motherboards/Z9PED8_WS/HelpDesk_CPU/

The fastest per core is the 2687w v2 but the chip with the highest number of cores is the 2697v2


----------



## halkx

The v3 chips use the 2011-3 socket so they are not physically compatible with our board I'm afraid


----------



## TheBlademaster01

E5-2697v2 is the fastest solution for this board.

Haswell uses a different pinout and is keyed differently so it won't fit physically in the 2011-0 sockets on this board.

E:

Ninja'd


----------



## RenoLevi

hmm cant find any ES E5-2697v2

Whats other higher core numbers are available for this board? and other 12 or 10 cores models?


----------



## RoboChimp

Accidentally snapped a USB port on my Z9, is a possible get it fixed or is the board stuffed?


----------



## Migonacci

Not sure if anyone has experienced this problem but here it goes...

The other day I was using my computer and it shutdown all of the sudden. When I tried to start the computer back up, the power button on the mobo and reset buttons would light up and stay on but nothing would actually power up. I tried to pull the power plug and switch the power supply off and on but still nothing..... the power button and reset would come on the mobo and nothing would actually turn on. I Also tried to troubleshoot the problem to see if it was the power supply but it was inconclusive...I tried to reset the cmos for the mobo etc etc. and nothing worked that day. I've had this issue happen before but only after trying to wake the computer from sleep. In that particular instance I would pull switch the power supply off and pull the plug and let the computer sit for a couple of minutes and then re plug and it would boot it up fine.

Anyway this morning I plugged the power cord back into the power supply, switched it on and hit the power button on the case and computer started right up and booted fine.... I ran OCCT benchmark on cpu for an hour just to check overall stability and everything was fine. I then shut down the computer just to see if it would boot back up and I ran into the same issue.... Does anybody have any insight as what maybe causing the issue?

I've had this configuration for over a year now and have only had this issue once in a while when trying to wake the computer from sleep. It seems like this might be some sort of weird incompatibility from the psu and mobo? Internal psu cables? (although I was running the comp fine for couple of hours) All other hardware components seem to be working fine...I checked RAM, gpu etc.

My specs:

Asus Z9
Dual E5 2670
64gb G.Skilll Ripjaws DDR3-1600
Seasonic X-1250 W
Gtx 295

Happy New Year everyone..


----------



## ZDngrfld

Check the CPU 8-pin power connectors. The X series power supplies I was using with my Z9 burnt up one of the 8-pins and that rig started acting like yours is...


----------



## aerosmith9110

My board just arrived







finally. I read on the manual that 4 ram slots are for each proc.. does it mean if say I have 1 proc only 4 ram slots would function even if I populated all 8 slots?

BTW anyone here wherein the whole board is watercooled? ie ek board WB ? I just want an idea on the connectors / fittings I need and how would the hose routing will be.


----------



## Migonacci

Quote:


> Originally Posted by *ZDngrfld*
> 
> Check the CPU 8-pin power connectors. The X series power supplies I was using with my Z9 burnt up one of the 8-pins and that rig started acting like yours is...


ZDngrfld,

The psu burnt one of the 8-pins on the mobo itself? I'm assuming you had to replace the motherboard.... but what about the psu? Was it just a bad psu unit or did you switch to another psu brand?

The cpu 8 pin power connectors on my board seem to be fine, I don't see any apparent damage to the pins. I was however using extension cables for the cpu and mobo power cables....so its likely they may have contributed.

I also tried to power up the system without the cpu power connectors and I was able to successfully get it to power up 4 times consecutively...until it exhibited the same problem. Just to eliminate the psu out of equation I powered up and down another older lower spec machine I have and everything seemed to be fine. Right now it seems like the culprit is the motherboard even though I can't definitively pin point the specific issue.


----------



## ZDngrfld

Quote:


> Originally Posted by *Migonacci*
> 
> ZDngrfld,
> 
> The psu burnt one of the 8-pins on the mobo itself? I'm assuming you had to replace the motherboard.... but what about the psu? Was it just a bad psu unit or did you switch to another psu brand?



I haven't done anything with it yet. I need to call Asus and RMA it. I had two X-650s have the same issue. I'm running a X-750 in this computer and I haven't had any issues. The plastic ends on the 8-pin cable were burnt on both the PSU and motherboard ends.


----------



## Migonacci

Quote:


> Originally Posted by *ZDngrfld*
> 
> 
> I haven't done anything with it yet. I need to call Asus and RMA it. I had two X-650s have the same issue. I'm running a X-750 in this computer and I haven't had any issues. The plastic ends on the 8-pin cable were burnt on both the PSU and motherboard ends.


Wow, that's pretty severe....yea my 8 pin connectors are aesthetically fine, nothing like the image you posted.


----------



## RoboChimp

I'm considering selling my Z9PE-D8 rig, If one of the USB ports doesn't work, am I still able to sell it?
Oh Crap the onboard firewire controller isn't working. Oh crap....









*Will anyone sell me a Z9PE-D8?*







You guys are going to 2011-3 anyway right?


----------



## LeMakisar

Quote:


> Originally Posted by *ZDngrfld*
> 
> 
> I haven't done anything with it yet. I need to call Asus and RMA it. I had two X-650s have the same issue. I'm running a X-750 in this computer and I haven't had any issues. The plastic ends on the 8-pin cable were burnt on both the PSU and motherboard ends.


I had the exact same problem with my first z9pe, it melted one of the 8pin CPU connectors ... at first my PC would just stop sometimes, and finally it didn't launch again









Asus sent me a replacement (used ...) one though I was in the 1st year of warranty.

A shame there was no alternative to this board. I'm also considering selling it now but I'm afraid that I wouldn't get much money for it, and even worse for the CPUs


----------



## aerosmith9110

how do you guys install win 7 with intel raid. Included drivers seems to give me error message saying driver needs to be signed.

update: it worked! I tried to load and win 7 is saying it needs to have signed driver.. but when I cancelled it the drives showed up!


----------



## LeMakisar

Hello again

I'm experiencing stuttering, in SLI or not, for example in Crysis 3.
I tried several PCIe ports (but the thing is I have a soundcard and a raid controller, so it's not easy to find room ...).
I did use 2 PCIe from the same cpu for my SLI setup.

Another thing I noticed is that sometimes my USB mouse just seem to go offline though leds are still on. I need to unplug/plug it to make it work again. Not sure it could be related in any way ...

I'm using latest bios, and retail CPUs

Any suggestion would be appreciated









Thanks !


----------



## fslharthy

Hello all,

am new to this forum, wish if someone can help me with my issue









My workstation freezes for strange reason, everything stucks mouse/keybaord cant move, screen stucked on last windows opened. and u cant do anything buy force shutdown the PC. it happens like 3-4 times a week running the workstation 24/7.

I tried messing with bios setting, but without luck









am running the latest bios: 5601

my setup:
Mobo: Asus Z9PE-D8
CPU: 2x E5-2697 v2
RAM: Corsair Vengeance LP 1866MHz
GPU: Nvidia GTX 780ti
Storage: 1TB Samsung 840 evo
PSU: Corsair AX1200i

anyone faced similar issue?

anyone can give me recommended bios setting?

thanks <3


----------



## Piccolo71320

Hi there

Because my English is not very good, unfortunately, this is a Google translation.









I had assembled to me about 2 1/4 years ago Z9PE-D8 WS with two E5-2687W ES (B0 stepping) for Foldinghome.
Because in the near future phase out the SMP BigWU's, I'm considering whether I should use the system in the future for gaming.

Now my question is what can I install in graphics cards, but I am bound Because of the ES with the B0 stepping to the 0503 bios:
With Nvidia is the way I've found GTX580 conclusion in this thread.

How it looks now with AMD?
Where is the line?
Did this unfortunately found nothing.

Thank you for your help









German text:


Spoiler: Warning: Spoiler!



Hallo zusammen

Da mein Englisch leider nicht besonders gut ist, ist dies eine Google-Übersetzung.

Ich hatte mir vor rund 2 1/4 Jahren ein Z9PE-D8 WS mit zwei E5-2687W ES (B0 Stepping) für Foldinghome zusammengebaut.
Da in naher Zukunft die SMP-BigWU's auslaufen, überlege ich mir ob ich das System in Zukunft auch für Gaming nutzen soll.

Jetzt meine Frage ist was ich an Grafikkarten einbauen kann, da ich Aufgrund der ES mit dem B0-Stepping an das 0503 Bios gebunden bin:
Mit Nvidia ist ja bei GTX580 Schluss so wie ich hier im Thread gefunden habe.

Wie sieht das Ganze nun mit AMD aus?
Wo ist da die Grenze?
Hab dazu leider nichts gefunden.

Vielen Dank für eure Hilfe


----------



## TheBlademaster01

Quote:



> Originally Posted by *LeMakisar*
> 
> Hello again
> 
> I'm experiencing stuttering, in SLI or not, for example in Crysis 3.
> I tried several PCIe ports (but the thing is I have a soundcard and a raid controller, so it's not easy to find room ...).
> I did use 2 PCIe from the same cpu for my SLI setup.
> 
> *Another thing I noticed is that sometimes my USB mouse just seem to go offline though leds are still on. I need to unplug/plug it to make it work again. Not sure it could be related in any way ...*
> 
> I'm using latest bios, and retail CPUs
> 
> Any suggestion would be appreciated
> 
> 
> 
> 
> 
> 
> 
> 
> 
> Thanks !


Oh, I have the same thing as that. Never knew it was related to the motherboard.

Which GPUs do you use and have you tried other drivers and/or SFR instead of AFR?

Quote:


> Originally Posted by *fslharthy*
> 
> Hello all,
> 
> am new to this forum, wish if someone can help me with my issue
> 
> 
> 
> 
> 
> 
> 
> 
> 
> My workstation freezes for strange reason, everything stucks mouse/keybaord cant move, screen stucked on last windows opened. and u cant do anything buy force shutdown the PC. it happens like 3-4 times a week running the workstation 24/7.
> 
> I tried messing with bios setting, but without luck
> 
> 
> 
> 
> 
> 
> 
> 
> 
> am running the latest bios: 5601
> 
> my setup:
> Mobo: Asus Z9PE-D8
> CPU: 2x E5-2697 v2
> RAM: Corsair Vengeance LP 1866MHz
> GPU: Nvidia GTX 780ti
> Storage: 1TB Samsung 840 evo
> PSU: Corsair AX1200i
> 
> anyone faced similar issue?
> 
> anyone can give me recommended bios setting?
> 
> thanks <3


What are your settings for RAM? Force 1600 or Force 1866?

Quote:


> Originally Posted by *Piccolo71320*
> 
> Hi there
> 
> Because my English is not very good, unfortunately, this is a Google translation.
> 
> 
> 
> 
> 
> 
> 
> 
> 
> I had assembled to me about 2 1/4 years ago Z9PE-D8 WS with two E5-2687W ES (B0 stepping) for Foldinghome.
> Because in the near future phase out the SMP BigWU's, I'm considering whether I should use the system in the future for gaming.
> 
> Now my question is what can I install in graphics cards, but I am bound Because of the ES with the B0 stepping to the 0503 bios:
> With Nvidia is the way I've found GTX580 conclusion in this thread.
> 
> How it looks now with AMD?
> Where is the line?
> Did this unfortunately found nothing.
> 
> Thank you for your help
> 
> 
> 
> 
> 
> 
> 
> 
> 
> German text:
> 
> 
> Spoiler: Warning: Spoiler!
> 
> 
> 
> Hallo zusammen
> 
> Da mein Englisch leider nicht besonders gut ist, ist dies eine Google-Übersetzung.
> 
> Ich hatte mir vor rund 2 1/4 Jahren ein Z9PE-D8 WS mit zwei E5-2687W ES (B0 Stepping) für Foldinghome zusammengebaut.
> Da in naher Zukunft die SMP-BigWU's auslaufen, überlege ich mir ob ich das System in Zukunft auch für Gaming nutzen soll.
> 
> Jetzt meine Frage ist was ich an Grafikkarten einbauen kann, da ich Aufgrund der ES mit dem B0-Stepping an das 0503 Bios gebunden bin:
> Mit Nvidia ist ja bei GTX580 Schluss so wie ich hier im Thread gefunden habe.
> 
> Wie sieht das Ganze nun mit AMD aus?
> Wo ist da die Grenze?
> Hab dazu leider nichts gefunden.
> 
> Vielen Dank für eure Hilfe


GTX 580/590 is the fastest you can use on the green side and HD 6970/6990 on the red side. You can use PCI-E 3.0 GPUs but you will need to switch to a non-stable BIOs for your CPUs, which I don't really recommend.


----------



## Piccolo71320

Quote:


> Originally Posted by *TheBlademaster01*
> 
> GTX 580/590 is the fastest you can use on the green side and HD 6970/6990 on the red side. You can use PCI-E 3.0 GPUs but you will need to switch to a non-stable BIOs for your CPUs, which I don't really recommend.


Thanks for the info.









If even with the AMD 6970 the Maximium is, the thing really is not worth it.








An unstable BIOS is not an option because I have all versions from 3404 already tested and none worked properly.

In that case I'll have to replace in the near future my 1090T PC by allowing faster.

The Z9PE-D8 WS I will then normal SMP projects fold and continue to use it as a DLNA server.

Thanks again for the help.

German text:


Spoiler: Warning: Spoiler!



Danke für die Info.

Wenn selbst bei AMD die 6970 das Maximium ist, lohnt sich die Sache nicht wirklich.
Ein unstabiles BIOS ist keine Option, da ich schon alle Versionen ab 3404 bereits durchgetestet habe und keines hat richtig funktioniert.

In dem Fall werde ich wohl in absehbarer Zeit meinen 1090T-PC durch was schnelleres ersetzten müssen.

Mit dem Z9PE-D8 WS werd ich dann normale SMP-Projekte falten und ihn weiterhin als DLNA-Server nutzen.

Nochmals Danke für die Hilfe.


----------



## LeMakisar

Quote:


> Originally Posted by *TheBlademaster01*
> 
> Oh, I have the same thing as that. Never knew it was related to the motherboard.
> 
> Which GPUs do you use and have you tried other drivers and/or SFR instead of AFR?


It's a couple of titan black
I'm updating Nvidia drivers every time a new one is available, and I must say the problem has been there for a while

I didn't modify settings at first, I'll give it a try. Wouldn't have imagined it's such a pain in the ass to have a playable game









edit : I forgot to mention, though applying the PCIe 3.0 patch, sometimes it reports one of my card is at 1.1, is this normal ?


----------



## aerosmith9110

anybody fitted this board with NZXT H630? Is it compatible?

- never mind. Got to purchase a MM ascension extended. But still it would be nice to know.


----------



## fslharthy

@TheBlademaster01:

thank you for replying bro <3

ok so i tried both 1600 and 1866:

with 1866 it cant go past windows loading logo on boot.
with 1600 it boots normal, but still got the freeze thing after couple of hours usage.

do u think this freeze thing is for sure RAM related? should i change the the whole set? what other settings i can test?


----------



## $ilent

Thread handed over to @TheBlademaster01


----------



## ocvn

Quote:


> Originally Posted by *fslharthy*
> 
> @TheBlademaster01:
> 
> thank you for replying bro <3
> 
> ok so i tried both 1600 and 1866:
> 
> with 1866 it cant go past windows loading logo on boot.
> with 1600 it boots normal, but still got the freeze thing after couple of hours usage.
> 
> do u think this freeze thing is for sure RAM related? should i change the the whole set? what other settings i can test?


Your configuration similar as mine. With auto ram configuration, your dram run at 1600, your corsair are 1866 so i think it wont be problem with your ram (My 1600 Nanya OC to 1900 with 1.65V run stable also). You try 1 stick per cpu first and run linX newest version with 48 threads and all ram. If you passed like 50 rounds, it should be fine. Check your vram voltage and timing also.


----------



## VitOne

Hello all! I am using Windows 8.1 and the latest BIOS. I never experienced problems but now I am noticing that my CPUs (2687W) are turbo boosting only at 3.4Ghz (34x) also if I am using one-threaded applications like Cinebench in 1-core bench mode. Can you help me? I tried different settings in the BIOS but nothing is working.


----------



## aerosmith9110

did any of you Water cooled the rams / chipset and both the processors for this board? I just need some idea on how to route the hose.


----------



## VitOne

Hello. I watercooled the chipset and the CPUs but not the RAMs.

EK waterblooks were really easy to install, I had no problems since April 2013 and I experienced lower sound, lower internal temperatures and optimal stability.


----------



## aerosmith9110

Tnx. Btw what case is that?


----------



## VitOne

Np. Define XL (Fractal). IMHO you could do a cleaner job integrating the pump in the reservoir and using some rigid tubes. Unfortunately I never had the time to do that, maybe for the next workstation.


----------



## aerosmith9110

My next upgrade on tubing would be ridgid tubes. But since my board is new and complementing in the future about new VCs figured pvc/advance lrt would be better for now


----------



## fslharthy

Quote:


> Originally Posted by *fslharthy*
> 
> Hello all,
> 
> am new to this forum, wish if someone can help me with my issue
> 
> 
> 
> 
> 
> 
> 
> 
> 
> My workstation freezes for strange reason, everything stucks mouse/keybaord cant move, screen stucked on last windows opened. and u cant do anything buy force shutdown the PC. it happens like 3-4 times a week running the workstation 24/7.
> 
> I tried messing with bios setting, but without luck
> 
> 
> 
> 
> 
> 
> 
> 
> 
> am running the latest bios: 5601
> 
> my setup:
> Mobo: Asus Z9PE-D8
> CPU: 2x E5-2697 v2
> RAM: Corsair Vengeance LP 1866MHz
> GPU: Nvidia GTX 780ti
> Storage: 1TB Samsung 840 evo
> PSU: Corsair AX1200i
> 
> anyone faced similar issue?
> 
> anyone can give me recommended bios setting?
> 
> thanks <3


I solved my problem by doing a clean Windows Install, and i thought it was Hardware related -.-

thanks all <3


----------



## VitOne

Quote:


> Originally Posted by *VitOne*
> 
> Hello all! I am using Windows 8.1 and the latest BIOS. I never experienced problems but now I am noticing that my CPUs (2687W) are turbo boosting only at 3.4Ghz (34x) also if I am using one-threaded applications like Cinebench in 1-core bench mode. Can you help me? I tried different settings in the BIOS but nothing is working.


Hello, I just wanted to share with you my experience. My problem is a software issue: Visual Studio 2013 Ultimate. Unfortunately it seems that the Hyper-V is creating issues with turbo-boost, allowing the OS to use just the lower boost frequency also in single-threaded tasks. I had to disable Hyper-V.


----------



## Piccolo71320

Hello everybody

A member of my F @ H team also has a Z9PE-D8 WS equipped with ES and wonders now whether the problem zutreffe with the 0503 BIOS well with him:
If the problem with the 0503 BIOS with the E5-2680 ES?
Does it concern only certain steppings?
If he can easily do a BIOS update or no longer run the E5-2680 ES?

Thank you for your help.

greeting

German text:


Spoiler: Warning: Spoiler!



Hallo Zusammen

Ein Mitglied von meinem [email protected] hat auch einen Z9PE-D8 WS mit ES bestückt und fragt sich nun ob die Problematik mit dem 0503-BIOS auch bei ihm zutreffe:
Besteht das Problem mit dem 0503-BIOS auch mit den E5-2680 ES?
Betrifft es nur bestimmte Steppings?
Kann er problemlos ein BIOS-Update machen oder laufen dann die E5-2680 ES nicht mehr?

Danke für eure Hilfe.

Gruss


----------



## TheBlademaster01

Quote:


> Originally Posted by *fslharthy*
> 
> I solved my problem by doing a clean Windows Install, and i thought it was Hardware related -.-
> 
> thanks all <3


Good to know, will keep this in mind for future reference








Quote:


> Originally Posted by *VitOne*
> 
> Hello, I just wanted to share with you my experience. My problem is a software issue: Visual Studio 2013 Ultimate. Unfortunately it seems that the Hyper-V is creating issues with turbo-boost, allowing the OS to use just the lower boost frequency also in single-threaded tasks. I had to disable Hyper-V.


Same as above.
Quote:


> Originally Posted by *$ilent*
> 
> Thread handed over to @TheBlademaster01


Thanks $ilent








Quote:


> Originally Posted by *Piccolo71320*
> 
> Hello everybody
> 
> A member of my F @ H team also has a Z9PE-D8 WS equipped with ES and wonders now whether the problem zutreffe with the 0503 BIOS well with him:
> If the problem with the 0503 BIOS with the E5-2680 ES?
> Does it concern only certain steppings?
> If he can easily do a BIOS update or no longer run the E5-2680 ES?
> 
> Thank you for your help.
> 
> greeting
> 
> German text:
> 
> 
> Spoiler: Warning: Spoiler!
> 
> 
> 
> Hallo Zusammen
> 
> Ein Mitglied von meinem [email protected] hat auch einen Z9PE-D8 WS mit ES bestückt und fragt sich nun ob die Problematik mit dem 0503-BIOS auch bei ihm zutreffe:
> Besteht das Problem mit dem 0503-BIOS auch mit den E5-2680 ES?
> Betrifft es nur bestimmte Steppings?
> Kann er problemlos ein BIOS-Update machen oder laufen dann die E5-2680 ES nicht mehr?
> 
> Danke für eure Hilfe.
> 
> Gruss


It is an issue related to steppings, not necessarily ES. Only B0 Sandy chips have this issue


----------



## Piccolo71320

@TheBlademaster01: Thanks for the info.


----------



## Migonacci

I was thinking of upgrading my current chips, E5 2670s ES, (C0 Stepping) to a pair of E5 2696v2 ES (QDKA) B1. Is anyone running these type of cpus on this board?

Based on Blademaster's bios compatibility chart it seems like the cpus should be compatible with bios rev 5103 up until 5404 and on.... (I'm running 65gb NON ECC ram)

I'm basically looking to increase the core count in my system for rendering purposes....

Any insight would be helpful.

Thanks


----------



## geort45

How much for those 2670s







? Are they V1 or V2? Thanks buddy!

Sent from my iPhone using Tapatalk


----------



## Migonacci

Quote:


> Originally Posted by *geort45*
> 
> How much for those 2670s
> 
> 
> 
> 
> 
> 
> 
> ? Are they V1 or V2? Thanks buddy!
> 
> Sent from my iPhone using Tapatalk


Geort45

They are v1 Sandy bridge. I'm not selling them just yet.... but I'll be sure to post them here once I get the replacement cpus.


----------



## LeMakisar

Hello again.
some time ago to my knowledge it was not possible to desync ram. For example 1600 max for a SBE. I saw posts that tend to show it's not the case anymore. I'm wondering since I have 2133 sticks .... and for now, still annoying stuttering that I'm eager to resolve








Thanks for the help

Edit : nevermind, after a few hours of struggle last night, a lot of clear cmos, I finally got 2133 working


----------



## Piccolo71320

Hello Everybody

Why the ES does not run with the B0 stepping with the newer BIOS versions?

Is it a hardware or a software problem?
Say it is for technical reasons due to a hardware problem of the B0 steppings or is it the BIOS itself because Asus has thrown the B0 from the list of supported CPUs.

Thanks for your info.

greeting

German text:


Spoiler: Warning: Spoiler!



Hello Everybody

Why the ES does not run with the B0 stepping with the newer BIOS versions?

Is it a hardware or a software problem?
Say it is for technical reasons due to a hardware problem of the B0 steppings or is it the BIOS itself because Asus has thrown the B0 from the list of supported CPUs.

Thanks for your info.

greeting


----------



## lucaskywalker

Hi,
I have this motherboard that works great with a pair of 2670 C0

I bought a pair of 2687 w C1 but does boot with only one CPU.
My bios is the last one: 5601

Has anyone managed to use theses CPUs?

May be with an older bios?

Regards


----------



## LeMakisar

Quote:


> Originally Posted by *ChrisSibbald*
> 
> I'm in!
> 
> And I have an issue I hope someone can help me with.
> 
> Here are my system specs:
> ASUS Z9PE-D8 WS MoBo AMI bios 503
> Dual Xeon 2687W CPU, each liquid cooled with Corsair Hydro H80
> Dual EVGA GTX 580
> 64GB Kingston ECC Registered DDR3 1300 RAM (8 x 8 GB)
> Enermax Platimax 1200 Watt PSU
> LSI 9620 4i 4 port SAS/SATA RAID controller
> 4 x 2TB Seagate Constellation ES
> Win 7 Ultimate
> 
> I run this box hard, crunching BOINC 24/7 full speed.
> 
> My issue:
> 
> I am getting a Windows WHEA event 19 frequently. This is a corrected machine check error. Details below:
> A corrected hardware error has occurred.
> 
> Reported by component: Processor Core
> Error Source: Corrected Machine Check
> Error Type: Bus/Interconnect Error
> Processor ID: 46
> 
> The details view of this entry contains further information.
> 
> I am not overclocking and have most bios settings to auto.
> 
> I turned off Speedstep in bios and the issue went away for four days. However, I want to know what is causing the issue and would like to be able to use speedstep.
> 
> Any ideas I can test?
> 
> Does anyone know the recommended voltage, frequency and timing settings for this CPU and RAM? I would like to try manual bios settings to see if I can get rid of or identify the cause of this issue.
> 
> Thanks in advance.
> 
> Cheers,
> Chris


Quote:


> Originally Posted by *ChrisSibbald*
> 
> Hi eq2account,
> 
> I am running a Z9 with two Xeon 2687w and 703 bios and the CPU runs at 3.4 GHz in turbo (see CPU-Z screenshot). Running fine but does still have the occasional WHEA Event 19 Warning that I have never been able to resolve.
> 
> Cheers,
> Chris


Hello,

I've noticed that I have also theseWHEA-logger 19 error code ... does anyone have any idea on how to solve this ?
Also, does anyone have an insight on the obscure bios settings like "mesegEN" for example ?

Thanks !


----------



## TheBlademaster01

Quote:


> Originally Posted by *Migonacci*
> 
> I was thinking of upgrading my current chips, E5 2670s ES, (C0 Stepping) to a pair of E5 2696v2 ES (QDKA) B1. Is anyone running these type of cpus on this board?
> 
> Based on Blademaster's bios compatibility chart it seems like the cpus should be compatible with bios rev 5103 up until 5404 and on.... (I'm running 65gb NON ECC ram)
> 
> I'm basically looking to increase the core count in my system for rendering purposes....
> 
> Any insight would be helpful.
> 
> Thanks


It should be fine in theory, I haven't personally tested it nor heard of anyone running it however. Do report back when you get them.
Quote:


> Originally Posted by *Piccolo71320*
> 
> Hello Everybody
> 
> Why the ES does not run with the B0 stepping with the newer BIOS versions?
> 
> Is it a hardware or a software problem?
> Say it is for technical reasons due to a hardware problem of the B0 steppings or is it the BIOS itself because Asus has thrown the B0 from the list of supported CPUs.
> 
> Thanks for your info.
> 
> greeting
> 
> German text:
> 
> 
> Spoiler: Warning: Spoiler!
> 
> 
> 
> Hello Everybody
> 
> Why the ES does not run with the B0 stepping with the newer BIOS versions?
> 
> Is it a hardware or a software problem?
> Say it is for technical reasons due to a hardware problem of the B0 steppings or is it the BIOS itself because Asus has thrown the B0 from the list of supported CPUs.
> 
> Thanks for your info.
> 
> greeting


Most likely a hardware issue. Coincidently B0 chips stop working reliably on BIOS versions that add PCI-E 3.0 support. They do POST from time to time, but fail like 2 out of 3 times to load the GPU VBIOS and then hangs in OpROM initialization stage (QCode "b2").
Quote:


> Originally Posted by *lucaskywalker*
> 
> Hi,
> I have this motherboard that works great with a pair of 2670 C0
> 
> I bought a pair of 2687 w C1 but does boot with only one CPU.
> My bios is the last one: 5601
> 
> Has anyone managed to use theses CPUs?
> 
> May be with an older bios?
> 
> Regards


Are you sure it is a C1 chip? Haven't heard of the issue. It might be a failure of the board as well.
Quote:


> Originally Posted by *LeMakisar*
> 
> Hello,
> 
> I've noticed that I have also theseWHEA-logger 19 error code ... does anyone have any idea on how to solve this ?
> Also, does anyone have an insight on the obscure bios settings like "mesegEN" for example ?
> 
> Thanks !


Usually this means a glitch in CPU or RAM.

This post
Quote:


> Originally Posted by *LeMakisar*
> 
> Hello again.
> some time ago to my knowledge it was not possible to desync ram. For example 1600 max for a SBE. I saw posts that tend to show it's not the case anymore. I'm wondering since I have 2133 sticks .... and for now, still annoying stuttering that I'm eager to resolve
> 
> 
> 
> 
> 
> 
> 
> 
> Thanks for the help
> 
> Edit : nevermind, after a few hours of struggle last night, a lot of clear cmos, I finally got 2133 working


Would make me think that the OC to 2133MHz wasn't fully stable. I personally have issues getting 1866MHz fully stable. Let me know if you solved this


----------



## LeMakisar

I was able to run stable at 2133, but it was long and painful








I just couldn't use "auto" settings to make it work.
Tested 24h with memtest86+ and 24h with memtest86 in the end, all OK

About the WHEA-logger error 19, as I tested the RAM I guess that would just leave the CPU, but I'm not sure what could be wrong ...
I've been searching for the voltage specs of 2687W without success, otherwise I'd try setting voltages instead of leaving the auto setting

I just hope my board isn't dying


----------



## Piccolo71320

Quote:


> Originally Posted by *TheBlademaster01*
> 
> Most likely a hardware issue. Coincidently B0 chips stop working reliably on BIOS versions that add PCI-E 3.0 support. They do POST from time to time, but fail like 2 out of 3 times to load the GPU VBIOS and then hangs in OpROM initialization stage (QCode "b2").


Thanks for the info.

Have still small ne OC question:
Has anyone managed that all cores all running at maximum turbo level ie at 33?
Had at least 210MHz clock more (I am running 105MHz BCLK stable) and that on 16 cores, it summert at F @ H.

Have A Beautiful Sunday

German text:


Spoiler: Warning: Spoiler!



Danke für die Info.

Hab noch ne kleine OC-Frage:
Hat es jemand hinbekommen das alle Kerne auf maximaler Turbostufe laufen sprich alle auf 33?
Wären immerhin 210MHz mehr Takt (bei mir läuft BCLK 105MHz stabil) und dass auf 16 Kerne summert es sich bei [email protected]

Schönen Sonntag


----------



## TheBlademaster01

Quote:


> Originally Posted by *Piccolo71320*
> 
> Thanks for the info.
> 
> Have still small ne OC question:
> Has anyone managed that all cores all running at maximum turbo level ie at 33?
> Had at least 210MHz clock more (I am running 105MHz BCLK stable) and that on 16 cores, it summert at F @ H.
> 
> Have A Beautiful Sunday
> 
> German text:
> 
> 
> Spoiler: Warning: Spoiler!
> 
> 
> 
> Danke für die Info.
> 
> Hab noch ne kleine OC-Frage:
> Hat es jemand hinbekommen das alle Kerne auf maximaler Turbostufe laufen sprich alle auf 33?
> Wären immerhin 210MHz mehr Takt (bei mir läuft BCLK 105MHz stabil) und dass auf 16 Kerne summert es sich bei [email protected]
> 
> Schönen Sonntag


I was curious about this as well. There was a rumour about someone running the highest turbo bin on all clocks with a 2687W and some Asrock X79 board, but I don't know how credible that is. I also heard something about unlocked Sandy-EP 8-core chips when I was building my Asus Z9 system, but in the end it turned out to be fake (Q19D IIRC).

Quote:


> Originally Posted by *LeMakisar*
> 
> I was able to run stable at 2133, but it was long and painful
> 
> 
> 
> 
> 
> 
> 
> 
> I just couldn't use "auto" settings to make it work.
> Tested 24h with memtest86+ and 24h with memtest86 in the end, all OK
> 
> About the WHEA-logger error 19, as I tested the RAM I guess that would just leave the CPU, but I'm not sure what could be wrong ...
> I've been searching for the voltage specs of 2687W without success, otherwise I'd try setting voltages instead of leaving the auto setting
> 
> I just hope my board isn't dying


Okay, well this board is not really tolerant for slight instabilities. I just leave my RAM at 1600MHz because anything higher can give glitches at the most annoying moments. Sometimes it even caused my system to fail loading applications in Windows. It might very well be the board though (VRMs overheating?). 2687Ws are pretty leaky chips at 150W, so it would be a point of concern.


----------



## LeMakisar

well it definitely dosn't smell good .. I had 5 crashes today when idle (strangely, nothing at full load)

Does someone have or know where I can find default 2687W voltages please ? I'd really like to set that manually and see what happens.

Thanks !


----------



## TheBlademaster01

Would make me think that something goes wrong when it enters C1E state. I'm not sure how you can adjust voltages for speedstep, but I think it's set too low


----------



## driftertravel

Hello all,
I'm trying to build a relatively cheap workstation and have picked up a z9pe-d8 ws for this purpose. I've been reading this thread a little, and, massive as it is, I have gotten the impression that I run some risks if I don't set everything up just right. I've always built my PCs but have never attempted a workstation build, so I could use some help.

For reference, the hardware I'm trying to use goes like this:
z9pe-d8 ws
2x e5 2687W v2, printed qe2t es 3.2ghz
8x 8gb Kingston KVR16E11/8 ddr3 1600mhz ecc ram (haven't purchased yet, let me know if there's a better option, I still have 8x8gb corsair vengeance 1600mhz non-ecc ram from my last build, and was kind of hoping there was a chance it would work... )
2x Samsung 850 EVO 1TB SSD (RAID0)
4x Seagate ST3000DM001 3TB Sata Hdd (2x RAID0, maybe 1xRAID10)
4x NVIDIA GTX 780ti 3GB Graphics
Corsair AX1500i 1500watt PSU

So my questions are thus:
1. Does anything seem amiss in the hardware lineup?
2. Is it correct that I should flash to BIOS 3109 and then to 5103 (but perhaps not beyond that?)
3. What else should I be concerned about/any advice?

I'd love to be able to take advantage of the experience you all have with these setups, so please let me know what you think.


----------



## Migonacci

Blademaster,

I got the E5 2695 v2s in the mail but on the chip itself its etched as 2.3 GHZ; on the intel e5 2600 v2 family list that same cpu is listed as 2.4 ghz......is the one I have 2.3 just because it wasn't qualified for retail yet? I still have to stress test these and see if they play nice with my system and mobo firmware.

http://ark.intel.com/products/series/75291/Intel-Xeon-Processor-E5-2600-v2-Product-Family#@Server

Also one other thing I noticed is that the E5 2696 v2 cpus are NOT listed on the e5 2600 v2 family....any ideas why this is? Are these just sold in business grade servers or something? I wasn't able to verify ....

Comparing the 2 processors , E5 2695 v2 and the 2696 v2 I also noticed that cpu world only delineates between single core turbo and all core turbo frequencies for the E5 2695....does this mean that the 2696 turbos @ 3.3 ghz for all cores as listed on there? Or they weren't able to verify this?

http://www.cpu-world.com/CPUs/Xeon/Intel-Xeon%20E5-2696%20v2.html

http://www.cpu-world.com/CPUs/Xeon/Intel-Xeon%20E5-2695%20v2.html

Thanks.
Quote:


> Originally Posted by *TheBlademaster01*
> 
> It should be fine in theory, I haven't personally tested it nor heard of anyone running it however. Do report back when you get them.


----------



## TheBlademaster01

Quote:


> Originally Posted by *Migonacci*
> 
> Blademaster,
> 
> I got the E5 2695 v2s in the mail but on the chip itself its etched as 2.3 GHZ; on the intel e5 2600 v2 family list that same cpu is listed as 2.4 ghz......is the one I have 2.3 just because it wasn't qualified for retail yet? I still have to stress test these and see if they play nice with my system and mobo firmware.
> 
> http://ark.intel.com/products/series/75291/Intel-Xeon-Processor-E5-2600-v2-Product-Family#@Server
> 
> Also one other thing I noticed is that the E5 2696 v2 cpus are NOT listed on the e5 2600 v2 family....any ideas why this is? Are these just sold in business grade servers or something? I wasn't able to verify ....
> 
> Comparing the 2 processors , E5 2695 v2 and the 2696 v2 I also noticed that cpu world only delineates between single core turbo and all core turbo frequencies for the E5 2695....does this mean that the 2696 turbos @ 3.3 ghz for all cores as listed on there? Or they weren't able to verify this?
> 
> http://www.cpu-world.com/CPUs/Xeon/Intel-Xeon%20E5-2696%20v2.html
> 
> http://www.cpu-world.com/CPUs/Xeon/Intel-Xeon%20E5-2695%20v2.html
> 
> Thanks.


You've probably got b1/es2 chips. Should not be a problem, but they are just clocked a tad lower than retail samples. It is possible that you've got 4P Ivy chips though (you can check this with HWInfo64). That would be a plus seeing as they can also be used in quad processer setup.

2696v2 is probably a limited fab by intel. They sometimes do this. They are retail and as such they come with warranty, but they are not part of the official 2600v2 family. You can actually get Intel to fab a custom chip for you if you order enough volume and the request is reasonable. I think there are some custom OEM and Apple chips as well. In the old Sandy Bridge range I remember E5-2689 being one of those custom chips.

@driftertravel

The only thing I'd be uncertain about would be the RAM, since that is kind of an issue with this board.


----------



## Migonacci

Quote:


> Originally Posted by *TheBlademaster01*
> 
> You've probably got b1/es2 chips. Should not be a problem, but they are just clocked a tad lower than retail samples. It is possible that you've got 4P Ivy chips though (you can check this with HWInfo64). That would be a plus seeing as they can also be used in quad processer setup.
> 
> 2696v2 is probably a limited fab by intel. They sometimes do this. They are retail and as such they come with warranty, but they are not part of the official 2600v2 family. You can actually get Intel to fab a custom chip for you if you order enough volume and the request is reasonable. I think there are some custom OEM and Apple chips as well. In the old Sandy Bridge range I remember E5-2689 being one of those custom chips.
> 
> .


Thanks for the info Blademaster, you're always a great wealth of information, I appreciate it.

It's unfortunate intel locks the multiplier to these chips.... I think worst comes to worst I'll try to up the baseclock for some mhz gains.

At the time I got the E5 2695 v2 the seller only had one e5 2696 v2 available...now it seems like they have more so I'm thinking it might be better to actually exchange to the e5 2696 v2, no?


----------



## TheBlademaster01

No problem. Yes, Intel locks down everything they can to increase sales. It depends on the difference in price whether it's worth it. I was thinking about getting some 2.3GHz 2695v2 myself, but they were more than I was willing to spend (like 1300 usd for the pair...).


----------



## Migonacci

Yes, thats about what I paid for my pair.....I think the diff in price is $130 for the 2.5ghz pair....I was just thinking in terms of possible resale value, the fact the 2956 v2 are retail and the extra boost in clock speed.....I still need to see how much oc I can get on the 2695. In the long run the difference in clock speed between the two is marginal but you know how these things are (ego)...you want to have the fastest possible lol. Also I will most likely sell my current ivy bridge e5 2670s to offset the cost.

Quote:


> Originally Posted by *TheBlademaster01*
> 
> No problem. Yes, Intel locks down everything they can to increase sales. It depends on the difference in price whether it's worth it. I was thinking about getting some 2.3GHz 2695v2 myself, but they were more than I was willing to spend (like 1300 usd for the pair...).


----------



## shadow001

Hi all, I've been running this board for nearly a year and something has me stumped for a while now....I recently got 4 video cards (XFX R290x 8GB), and have tried all 4 cards separately and they all work fine on the board, but try as I might I can't get more than 3 cards to work in crossfire mode...I'm currently running bios 5404 on the board and the latest drivers for the video cards.

Is there a specific setting in the bios I may be missing, or perhaps a jumper on the motherboard that needs to set that allows windows to see that last card?

Thanks.


----------



## shadow001

I've read the manual again, and there's a jumper on the board, that controls PCI-e slots 1-3-5-7 which are the slots that the cards are plugged into basically, and the manual is a little light on the details and only mentions this jumper is related to the SM bus connected to the PCH (this is the default setting) or connected to the BMC, which I have no clue what that is really...

Guess i'll just try it and if nothing changes, set it back to default.


----------



## shadow001

No change...It was actually worse as it disables the DVI outputs when set in the other position, so back to home base it goes..


----------



## ZDngrfld

Quote:


> Originally Posted by *shadow001*
> 
> Hi all, I've been running this board for nearly a year and something has me stumped for a while now....I recently got 4 video cards (XFX R290x 8GB), and have tried all 4 cards separately and they all work fine on the board, but try as I might I can't get more than 3 cards to work in crossfire mode...I'm currently running bios 5404 on the board and the latest drivers for the video cards.
> 
> Is there a specific setting in the bios I may be missing, or perhaps a jumper on the motherboard that needs to set that allows windows to see that last card?
> 
> Thanks.


This is what I remember about AMD cards and this board:
Quote:


> Originally Posted by *TheBlademaster01*
> 
> The top two run from CPU 0 and the last two run from CPU 1. It's best to have both GPUs connected to the same host because of latency. I know that Radeon R9 290 series even stutter when they aren't connected to the same host in games due to the fact that AMD runs crossfire via the DMA engine.


----------



## shadow001

Quote:


> Originally Posted by *ZDngrfld*
> 
> This is what I remember about AMD cards and this board:


Not following you?...I'm using 4 identical cards and i understand that one pair is running of the PCI-e controller of CPU 1, and the other pair off of PCI-e controller of CPU 0, but i'm still missing one of the cards....Windows doesn't see it at all, but i know all 4 cards work fine when i run them separately.

I'm not just using 2 cards basically.


----------



## ZDngrfld

Quote:


> Originally Posted by *shadow001*
> 
> Not following you?...I'm using 4 identical cards and i understand that one pair is running of the PCI-e controller of CPU 1, and the other pair off of PCI-e controller of CPU 0, but i'm still missing one of the cards....Windows doesn't see it at all, but i know all 4 cards work fine when i run them separately.
> 
> I'm not just using 2 cards basically.


Ahh, I didn't understand that Windows wasn't seeing the fourth card at all. Have enough power for everything?


----------



## shadow001

Quote:


> Originally Posted by *ZDngrfld*
> 
> Ahh, I didn't understand that Windows wasn't seeing the fourth card at all. Have enough power for everything?


Twin 1200 watt PSU's with the loads split evenly between both, so they're not stressed anywhere close to their limit really, and I see the fans on all 4 cards starting up, there's no beeps or error codes displayed at all....I'll check the bios again to see if it's detected, but if it were then windows should see it too, and it doesn't.

Checked all the jumpers on the motherboard and got nothing there, so unless there's a setting in the bios i'm missing, it might be an incompatibility problem in the bios.....I might fire an Email to Asus support to see if they've see this issue....Seems the board runs fine with 4 way SLI.


----------



## ZDngrfld

Quote:


> Originally Posted by *shadow001*
> 
> Twin 1200 watt PSU's with the loads split evenly between both, so they're not stressed anywhere close to their limit really, and I see the fans on all 4 cards starting up, there's no beeps or error codes displayed at all....I'll check the bios again to see if it's detected, but if it were then windows should see it too, and it doesn't.
> 
> Checked all the jumpers on the motherboard and got nothing there, so unless there's a setting in the bios i'm missing, it might be an incompatibility problem in the bios.....I might fire an Email to Asus support to see if they've see this issue....Seems the board runs fine with 4 way SLI.


Definitely not lacking in power! This might be a long shot, but do you have a 4-pin molex plugged into the EZ_PLUG connector? Give that a shot if not. It's supposed to send supplemental power to the PCI-e slots


----------



## shadow001

Quote:


> Originally Posted by *ZDngrfld*
> 
> Definitely not lacking in power! This might be a long shot, but do you have a 4-pin molex plugged into the EZ_PLUG connector? Give that a shot if not. It's supposed to send supplemental power to the PCI-e slots


It's worth a try for sure....

If that doesn't work, i'll try a different approach in that while I know all the cards work when tested one by one, i'll add one card at a time and see from which motherboard slot is one of the cards not being detected.....I can only be slot 3, slot 5 or slot 7.

Then i'll see in my old stock if I have a spare card I can plug in, not necessarily a video card mind you, but something else like a sound card just to see if windows can see it after it's booted up, so that I know the slot works at least and it has to be down to a bug in the bios most likely.


----------



## shadow001

The manual doesn't really say anything when it comes to that 4 pin molex connector, but I suppose 4 of these cards do qualify as high power usage devices...









Also checked the manual for anything I may have missed on the crossfire front, and the funny thing there is that it provides instructions on how to setup just a 2 card configuration, but funny enough that for SLI, it explains all for a 3 and a 4 card SLI setup...


----------



## shadow001

Interesting....I tried the 4 pin molex and once it's plugged in, and that was a connector powering up a water pump for the water loop which I know works, the system does not boot up at all.....Nothing happens really, and once I unplugged it it's starts back up just fine.

Is there any setting in the bios that enables that molex or any other step I might have missed?


----------



## shadow001

It's not the cards either as all 4 work and I tried the sound card in the slot that wasn't detecting the 4th video card and windows sees it, so the slot itself is working....Looks more and more like a bios bug that can handle up to 3 cards, but not a 4th one in crossfire.

Still have no clue why plugging in the molex makes the system not boot, and it boots fine with it unplugged..?


----------



## shadow001

I think I have an idea why it doesn't boot with the 4 pin molex.....I was using one that was connected to the modified PSU that primarily powers up the video cards, and it's either completely off or on.

What i'll do is use one from the regular non modified PSU, which is the one that powers the motherboard with the 24 pin ATX, and the pair of 8 pin power connectors, so that all motherboard power comes from the same PSU.


----------



## driftertravel

Just wanted to report that the build I posted about a few posts back went off without a hitch, I literally threw everything in the case, powered up, setup RAID arrays, installed Windows 10 and drivers and everything worked perfectly. Occasionally Intel LSI RAID won't recognize those arrays on boot, which is weird, but a restart fixes that. I was a little worried reading this thread that it'd be a nightmare getting everything to work, so I wanted to post something encouraging for people thinking about a similar build. It's literally 3x faster than my last i7 computer at the same frequency in Cinebench and Photoscan testing. Cinebench R15 is running at 2176 CPU and 60.38FPS OpenGL.

I wanted to ask a few questions:

1. This is the place where people run similar rigs, do those Cinebench scores seem like what I should be getting?

2. Where should I start overclocking this thing? I have BCLK at 105 and it's stable, I've tried pushing the multiplier to 34 from 32 but I honestly don't see any change in average frequency while running tests with CPU-Z open. Everything seems stable running Prime95, but that may be because I'm not overclocking it correctly... Is there anything I have to do when overclocking in the BIOS to get it to take? Is it even worth pushing this thing? I'm happy with the performance so far, but better is, well... better.

3. Does anyone have any tips on getting 2133 ram to work at that speed? Some of the work that I do is really dependent on RAM speed, so I'm going to try 64GB of quad channel G.SKILL Ripjaw-Z 2133 (which is on the approved vendors list at that speed) to see if I can get it to work instead of the Corsair 1600mhz RAM which is running at triple channel...

4. Just tested my RAID0 speeds, and I appear to only be getting 6gbps speeds off the Marvell RAID SSDs, do I need to force SATAIII in BIOS or something to get 2x SATAIII speeds?

Thanks for the helpful thread!


----------



## shadow001

Tested everything out and all the and all the cards work, flashed the bios successfully and the driver package now installs properly, so we're down to either a limitation in the Bios that only supports crossfire up to 3 cards, or there's bios settings not listed that might help detect that 4th card.

Fired an Email to tech support and detailed everything I did, and they'll contact me within 48 hours and we'll see from there.


----------



## aerosmith9110

Hi,

What commonly used benchmarks do you recommend for multi - processors and multi cores ( 8 core 16 threads x2 ) to show case vs single core processors multi thread? Basically, Justifying the price.

Thanks!


----------



## Migonacci

Blademaster,

I dropped in the 2 e5 2695 v2 chips in my system and the system booted up fine. I'm running bios version 5601.

However.....when I started to benchmark the chips I noticed that all the hardware monitor apps, (core temp, speccy64, cpuz etc) were showing the chips running at 2.6 ghz at full load and turbo boosting to 2.8 under normal load conditions. The multiplier is obviously locked at 26 @ 100 mhz baseclock.....so what do you think is going on here? The chip itself shows up to be labeled as 2.3ghz .....OCCT benchmark showed it as a 13% overclock....

Is this some sort of custom chip by intel?....kind of like how you mentioned the 2696 v2 to be? Or are the hardware monitors way off?

See the hardware info here 

Thanks.


----------



## ocvn

Quote:


> Originally Posted by *driftertravel*
> 
> Just wanted to report that the build I posted about a few posts back went off without a hitch, I literally threw everything in the case, powered up, setup RAID arrays, installed Windows 10 and drivers and everything worked perfectly. Occasionally Intel LSI RAID won't recognize those arrays on boot, which is weird, but a restart fixes that. I was a little worried reading this thread that it'd be a nightmare getting everything to work, so I wanted to post something encouraging for people thinking about a similar build. It's literally 3x faster than my last i7 computer at the same frequency in Cinebench and Photoscan testing. Cinebench R15 is running at 2176 CPU and 60.38FPS OpenGL.
> 
> I wanted to ask a few questions:
> 
> 1. This is the place where people run similar rigs, do those Cinebench scores seem like what I should be getting?
> 
> 2. Where should I start overclocking this thing? I have BCLK at 105 and it's stable, I've tried pushing the multiplier to 34 from 32 but I honestly don't see any change in average frequency while running tests with CPU-Z open. Everything seems stable running Prime95, but that may be because I'm not overclocking it correctly... Is there anything I have to do when overclocking in the BIOS to get it to take? Is it even worth pushing this thing? I'm happy with the performance so far, but better is, well... better.
> 
> 3. Does anyone have any tips on getting 2133 ram to work at that speed? Some of the work that I do is really dependent on RAM speed, so I'm going to try 64GB of quad channel G.SKILL Ripjaw-Z 2133 (which is on the approved vendors list at that speed) to see if I can get it to work instead of the Corsair 1600mhz RAM which is running at triple channel...
> 
> 4. Just tested my RAID0 speeds, and I appear to only be getting 6gbps speeds off the Marvell RAID SSDs, do I need to force SATAIII in BIOS or something to get 2x SATAIII speeds?
> 
> Thanks for the helpful thread!


1. Yes, cinebench both 11 and 15 versions. Beside that
2. You need to work with PCI-E gen/ HW power in the bios for overclocking. E5-2600 lock multiplier, so you only can change bclk atm. However, when changing bclk, it will affect all other components like PCI-E, Sata, USB etc... ALso nvidia is better dedicated VGA than AMD when you jump up the BCLK.
3. I tried ram from 1333 to 1900, beside AIDA 64, no difference in other application.
4. You can't. internal sata limited to 6GBps. Try dedicated raid card.
Quote:


> Originally Posted by *aerosmith9110*
> 
> Hi,
> 
> What commonly used benchmarks do you recommend for multi - processors and multi cores ( 8 core 16 threads x2 ) to show case vs single core processors multi thread? Basically, Justifying the price.
> 
> Thanks!


Cinebench/ LinX 64 avx etc...
Quote:


> Originally Posted by *Migonacci*
> 
> Blademaster,
> 
> I dropped in the 2 e5 2695 v2 chips in my system and the system booted up fine. I'm running bios version 5601.
> 
> However.....when I started to benchmark the chips I noticed that all the hardware monitor apps, (core temp, speccy64, cpuz etc) were showing the chips running at 2.6 ghz at full load and turbo boosting to 2.8 under normal load conditions. The multiplier is obviously locked at 26 @ 100 mhz baseclock.....so what do you think is going on here? The chip itself shows up to be labeled as 2.3ghz .....OCCT benchmark showed it as a 13% overclock....
> 
> Is this some sort of custom chip by intel?....kind of like how you mentioned the 2696 v2 to be? Or are the hardware monitors way off?
> 
> See the hardware info here
> Thanks.


2.3 is base clock for your chip. 2.6 all core boot and 2.8 is single core boot. like 2697v2. Base clock 2.7, turbo all 3GHz and single core 3.5GHz.


----------



## Migonacci

Quote:


> Originally Posted by *ocvn*
> 
> 2.3 is base clock for your chip. 2.6 all core boot and 2.8 is single core boot. like 2697v2. Base clock 2.7, turbo all 3GHz and single core 3.5GHz.


Ah, you're right OCVN, I wasn't paying attention closely. I just assumed it would default to its baseclock at all cores given the high count of cores....but like you mentioned, the spec says otherwise. Thanks for pointing it out. Now I just have to see how much I can get out of it by upping the baseclock a bit.


----------



## squidman

Hi sorry for hijacking...

Still on v1 ES E5-2665's, BCLK at 105 seems pretty stable but pretty useless (still getting around 17.xx on cinebench, and Taobaoing see what seem to be pretty decent E5-2697 v2's for around 3880 yuan a piece. However, of course they are getting scarce, so and sandy bridge etc not to mention ivy is getting pretty obsolete.

So to upgrade, I could put down around 8000 yuan for a couple of Ivy B0-B1 e5-2697 v2's, or the other option is a more future looking upgrade (say the supermicro board with pci-e slots spaced so that you don't lose any slots), + more taobao ES chips + ddr4 ram. All of which would be a lot more expensive, but this current platform seems to be getting obsolete a bit too fast. The best I could hope for on current platform (work with 3d etc) would be a cinebench 11.5 score of around 30-32, whereas the newer ES chips V3 for cheaper are already getting around 40! Of course the 500-700 bucks for a new mobo and ditto 32-64 gb ram sucks! However taobao is rife with v3 chips for cheap at the moment.

Any ideas? What are you guys doing vis-a-vis upgrades? Looking at building a DAS (no space in my corsair 900d for more hdd's!), so guess could use existing system for both rendering and file server (one of those DIY Norco 24 bay jobbies)...somehow even with 32? tb of useable space have only 6 tb left...damn! (backup of backup of backup! Can't be too paranoid! and then double backup to cloud at least 2 places!)









Wow less than 2 years old and already obsolete! However was a pretty good upgrade from my laptop! Looking better now that I pimped it with some cheap rgb led's, but cable management...oh well....







will get around to it.

:thumb

:


----------



## squidman

Piccolo71320,

You can flash to a newer bios (IFF!!! you follow instructions carefully) even using B0 ES chips...make sure not to miss any steps. I'm using b0 e5-2665's and on the first .cap bios (whatever that was...can't find it would have to reboot). However, on ES chips, you need to read the thread carefully! Otherwise things can go wrong really fast...yes all 193 pages of it! Go one bios too far and you're looking at ordering a custom bios chip (not the end of the world, around 11 euro!) to get your system working again. In fact, I would consider doing that anyway (bios chips flashed with various versions) but so far my ***** has been (sort of!) stable. (apart from hdd's dying, but that's another thing, especially with 15 of them!). NB!! you need to flash in a certain very particular order! The oldest one for me was stable, then I had some unstable ones, but current one (first .cap) has been stable for a year, albeit, graphics cards don't want to work on the blue (16x 3.0) pci-e slots. GTX 770. However only losing 1-2 fps gaming so no loss.

Sooo...to recap...Gtx 770 (now obsolete) works fine for me, and no doubt 980/970 will also work. So no problem gaming. Your only problem will be, once you buy one of those 4k screens which are becoming cheaper...damn not even sli 980 get good framerates!









Edit: Still running prime95 after boosting BCLK to 105 (and my room gets hot!) but system seems pretty stable.temps of cpu cores around 45-50...stupid CPUTIN on CPUID monitor reading 124 C (must be false reading), and drawing around 390 watts at the wall. This after around 45 mins of prime95..

Those all-in-one coolers (have 2 x h80i's) are prbly overkill for xeons...but liking the temps! However Corsair fans are friggin noisy! Would have replaced them long ago, but the price tag of 7 x noctua fans stopped me from doing do!


----------



## squidman

Quote:


> Originally Posted by *Migonacci*
> 
> Blademaster,
> 
> I dropped in the 2 e5 2695 v2 chips in my system and the system booted up fine. I'm running bios version 5601.
> 
> However.....when I started to benchmark the chips I noticed that all the hardware monitor apps, (core temp, speccy64, cpuz etc) were showing the chips running at 2.6 ghz at full load and turbo boosting to 2.8 under normal load conditions. The multiplier is obviously locked at 26 @ 100 mhz baseclock.....so what do you think is going on here? The chip itself shows up to be labeled as 2.3ghz .....OCCT benchmark showed it as a 13% overclock....
> 
> Is this some sort of custom chip by intel?....kind of like how you mentioned the 2696 v2 to be? Or are the hardware monitors way off?
> 
> See the hardware info here
> 
> Thanks.


Looking at taobaoing same chips - e5-2695 v2's - (around 1000 yuan cheaper than e5-2697 v2's), what are you doing on say cinebench 11.5? Still can't get my head around cinebench r15 (getting old...like keeping up with 3d mark scores!)...unfortunately most of the 2695's seem to be low ghz...and I've only 'flea-bayed' so far not actually purchased from Taobao (do I REALLY? need a taobao agent?)..









I really need to see around 30-34 on Cinebench 11.5 when spending 2k usd taobaoing, otherwise it may be safer and more future-proof to just go the new v3 platform...


----------



## squidman

Our Asus was the Schiidt! back in the day, and the new Asus z10pe-d16? or whatever looks the goods, but there's really a new proper replacement for our board which just schiiidts all over asus....check it out!

http://www.servethehome.com/Server-detail/supermicro-x10drg-q-review-gpu-compute-server-motherboard/

http://www.supermicro.com/products/motherboard/Xeon/C600/X10DRG-Q.cfm

Yeah a supermicro but with all pci-e spaced out so you don't lose any slots to double spaced cards! Storage is meh! but who cares, slot in a raid card and good to go!

NB not an advertisement for Schidt! audio products!


----------



## ocvn

Quote:


> Originally Posted by *squidman*
> 
> I really need to see around 30-34 on Cinebench 11.5 when spending 2k usd taobaoing, otherwise it may be safer and more future-proof to just go the new v3 platform...










DF setting, no OC with [email protected], I got around 33 pts on cinebench 11.5. with your 2695 v2, I think you could get around 30 pts in cinebench 11.5


----------



## squidman

Haven't pulled the trigger yet ocvn! What to do with those 2665's tho? Can't sell them for much. Just enjoying the disco lights from the led rgb's I picked up today, really sets off the 900d, should have done it a long time ago (if only to be able to see inside without a headleamp!)..









But then again, all those cheap taobao 12 and 14, 16 v3 core es doing 40+ in cinebench (and my render times in vray!) makes me wonder...









Edit: OCVN at 3 ghz you're not on el-cheapo taobao ES 2697's are you? the highest I've seen is around 2.5 -2.6 with all turbo up to around 2.8 ghz there...and for around 8k yuan you can get an ES chip at 2.8 ghz...out of my budget...


----------



## aerosmith9110

Hi All,

Did anyone of you able to change the Fan speed via software in windows 7?


----------



## squidman

Hmmm, using dog**** corsair link (first version!) still here...

Not to hijack thread too much, but am doing a simple AE clip (imported a 12 minute skiing clip and just put on one LUT ...gopro vid, so just putting on the one protune lut to compensate as I filmed in protune...). This off a very old gopro hero2 in 1080p. Wow, is it me or does AE like a lot of ram? 12 minute clip with simple color correction, and already up to 25? gb? ***? Not too mention, cpu usage is like friggin zero? And 12 minute clip will take like 2-3 hrs to color correct ? ***? On a dual-xeon workstation?

OK sorry for that, thought there might be ors out there who have experienced (gopro studio just crashes or doesn't even import vids!) or just crzy-slow exports in Adobe CC. Wow thought we were in the time of lightning-quick rendering by now, guess I was wrong! My 2 x E5-2665's are pretty much dog**** I know, but they still equal the top of the line current i7! Is it me or are software companies just getting worse at utilizing multi-threading and gpgpu?


----------



## scum

thinking about upgrading to 2697 v2 from 2640 v2 is the performance vs cost worth it ?


----------



## squidman

What is the main usage scenario`? For example, Premiere Pro CChardly uses 10% of available threads when rendering, whereas rendering in Max/Maya...can't have too many cores! At least that's my case....for multithreaded apps 2x E5-2697 v2's get around 30 on Cinebench 11.5 (I get around 17 on 2x E5-2665 ES)...
some apps prefer higher frequency, others do better with more cores.


----------



## scum

i use various video editing software premiere etc , was going to sell off my x99 system as use this system for gaming too

now the search is on for some reasonably priced 2697 v2


----------



## celemine1gig

Hi guys,

at the moment I own two Z9PE-D8 boards.

One of them suffers from the issue of disappearing RAM (either reported as "0 MB", or the full slot is reported as "disabled"). In addition it is not always the same slot that fails like that, but this can change from one (re-)boot to the other.
Read about this before and this probably means that the board needs to be replaced. Right? Maybe some comments from the people who have had this board a while longer than me?









The other board does not exhibit THAT weird RAM problem, but just like the other one it does not like the usage of all memory slots. I have a total of 80GB ECC Registered RAM here at the moment.
- 4x 4GB Hynix DDR3-1333 ECC reg
- 8x 4GB Samsung DDR3 ECC reg
- 8x 4GB Hynix DDR3-1600 ECC reg

All the RAMs seem to work fine, but as soon as I try to use 4 dimm slots per CPU, the board starts to act up with instabilty? I already tried a little more vdimm, relaxed timings and also more VSSA, but so far to no avail. I have both a retail Xeon 2603 V2 here and some ES C0 Sandies. Regardless of the CPU, the RAM problems persist. Any tips on that from the Z9PE-D8 pros?

Thank you in advance!

It is starting to get on my nerves to continuesly have to debug things with this board. I only bought the second Z9PE-D8 to be able to troubleshoot the issues with the first. This is not only starting to be a very expensive build, but also one that takes a LONG time to finish with the result of a stable and usable system.


----------



## squidman

Scum:

I've seen e5-2697 v2 on Taobao for around 4-5000 yuan (ES version), however they're typically clocked fairly low (2.3ghz or lower). You may want to look at e5-2687w v2 for video editing (premiere seems to prefer higher clock rates as opposed to more cores). Or another of the high ghz v2's....


----------



## scum

Quote:


> Originally Posted by *squidman*
> 
> Scum:
> 
> I've seen e5-2697 v2 on Taobao for around 4-5000 yuan (ES version), however they're typically clocked fairly low (2.3ghz or lower). You may want to look at e5-2687w v2 for video editing (premiere seems to prefer higher clock rates as opposed to more cores). Or another of the high ghz v2's....


was thinking going with e5 2667 v2, but 2687 seem too be more available


----------



## aerosmith9110

Hi theblademaster01,

The List of xeon compatibility on the first page.... are these all tested? I mean as there are no notes its all good?


----------



## 82ghost82

Quote:


> Originally Posted by *celemine1gig*
> 
> Hi guys,
> 
> at the moment I own two Z9PE-D8 boards.
> 
> One of them suffers from the issue of disappearing RAM (either reported as "0 MB", or the full slot is reported as "disabled"). In addition it is not always the same slot that fails like that, but this can change from one (re-)boot to the other.
> Read about this before and this probably means that the board needs to be replaced. Right? Maybe some comments from the people who have had this board a while longer than me?
> 
> 
> 
> 
> 
> 
> 
> 
> 
> The other board does not exhibit THAT weird RAM problem, but just like the other one it does not like the usage of all memory slots. I have a total of 80GB ECC Registered RAM here at the moment.
> - 4x 4GB Hynix DDR3-1333 ECC reg
> - 8x 4GB Samsung DDR3 ECC reg
> - 8x 4GB Hynix DDR3-1600 ECC reg
> 
> All the RAMs seem to work fine, but as soon as I try to use 4 dimm slots per CPU, the board starts to act up with instabilty? I already tried a little more vdimm, relaxed timings and also more VSSA, but so far to no avail. I have both a retail Xeon 2603 V2 here and some ES C0 Sandies. Regardless of the CPU, the RAM problems persist. Any tips on that from the Z9PE-D8 pros?
> 
> Thank you in advance!
> 
> It is starting to get on my nerves to continuesly have to debug things with this board. I only bought the second Z9PE-D8 to be able to troubleshoot the issues with the first. This is not only starting to be a very expensive build, but also one that takes a LONG time to finish with the result of a stable and usable system.


Hi,
I had the same problem with ECC 1333 MHz ram (Samsung), but I was trying to overclock the frequency to 1600 MHz; if you are trying to overclock the ram too, go back to original frequency.


----------



## celemine1gig

That would make sense.

Problem is the following:
On my "good" board I have a test running at the moment. Key point is that I only have RAM in the two leftmost slots for each CPU. And I can run the Samsung DDR3-1333 reg ECC modules at a stable DDR3-1680 (105 MHz bus), while set to timings of 11-11-11-28.
Test ran several times already and seems to be solid.

However, if I put RAM in the slots to the right of each CPU, I get complete instability. Even at underclocked speeds of DDR3-1066 for the RAM. Always ECC errors. So single bit errors, which get corrected, but slow everything down significantly!!!
As I said, all RAM modules are 100% fine. That was tested with day long runs of Passmark Memtest 6.00 (with full ECC support!).

So this could only be a CPU or motherboard problem.
CPUs I got two kinds. A dual setup with ES Sandy CPUs, and a retail E5-2603 V2. Both behave exactly the same. So this would be a HUGE coincidence. Especially considering, that the retail CPU is Ivy Bridge based, while my ESs are Sandies.

I am almost certain that Asus messed something up here.
I mean:
- 3 different types of DDR3 ECC reg modules
- 2 different types of CPUs
- ALWAYS the same ECC errors with 4 modules

That looks bad for ASUS and their Z9PE-D8. Now, some poeple might say: "The RAM might not be on the compatibility list!"
My answer would be: That might be true, but Samsung and Hynix are not really known to produce crappy server RAMs, so a board should be able to at least run it stable at stock speeds. And the Z9PE-D8 does not seem to be up to the task, which is just sad.


----------



## Piccolo71320

Quote:


> Originally Posted by *squidman*
> 
> Piccolo71320,
> 
> You can flash to a newer bios (IFF!!! you follow instructions carefully) even using B0 ES chips...make sure not to miss any steps. I'm using b0 e5-2665's and on the first .cap bios (whatever that was...can't find it would have to reboot). However, on ES chips, you need to read the thread carefully! Otherwise things can go wrong really fast...yes all 193 pages of it! Go one bios too far and you're looking at ordering a custom bios chip (not the end of the world, around 11 euro!) to get your system working again. In fact, I would consider doing that anyway (bios chips flashed with various versions) but so far my ***** has been (sort of!) stable. (apart from hdd's dying, but that's another thing, especially with 15 of them!). NB!! you need to flash in a certain very particular order! The oldest one for me was stable, then I had some unstable ones, but current one (first .cap) has been stable for a year, albeit, graphics cards don't want to work on the blue (16x 3.0) pci-e slots. GTX 770. However only losing 1-2 fps gaming so no loss.
> 
> Sooo...to recap...Gtx 770 (now obsolete) works fine for me, and no doubt 980/970 will also work. So no problem gaming. Your only problem will be, once you buy one of those 4k screens which are becoming cheaper...damn not even sli 980 get good framerates!
> 
> 
> 
> 
> 
> 
> 
> 
> 
> Edit: Still running prime95 after boosting BCLK to 105 (and my room gets hot!) but system seems pretty stable.temps of cpu cores around 45-50...stupid CPUTIN on CPUID monitor reading 124 C (must be false reading), and drawing around 390 watts at the wall. This after around 45 mins of prime95..
> 
> Those all-in-one coolers (have 2 x h80i's) are prbly overkill for xeons...but liking the temps! However Corsair fans are friggin noisy! Would have replaced them long ago, but the price tag of 7 x noctua fans stopped me from doing do!


Hello Squidman

Speeches from operation with one or two ES Xeons with B0 stepping?

With only a Xeon I get the 5701 to run stable, but not with two Xeon's.

The problem is that I do not have access to the PCI-E Xeons settings for the second and so could set this on Gen2.


----------



## squidman

Uhhhm...the automatic translation didn't work too well there! (Ich spreche ziemlich Null Deutsch!)









Edit: I'm on BIOS 3507 (I think!) or at any rate the very first .cap bios, which is 'sort of' stable even with B0 chips. I've never gotten pci-e 3.0 to work, but don't think it really matters....difference between gen 1,2,3 of pci-e on applications is negligable. Worse is some slots not working with my gtx 770 (all the faster blue ones....).

Dunno if that helps









Edit 2: There was a mention earlier in the thread of a Nvidia update for pci-e 3.0 which I haven't tried. I thought sandy chips didn't support this originally (was introduced with Ivy Bridge and xeon v2's). Again, don't think it has much effect on performance...

Edit 3: No time to go and search for your original post, but if you're on sandy bridge b0, and on BIOS 5701, then you've probably gone several BIOS generations too far. Can be fixed by ordering custom BIOS chips for around 20 euros...google for suppliers. Edit, while you're at it, order 2, both flashed with the earliest BIOS for the mobo...then take the one as far as it will go, and iff you go one BIOS update too far, you know where to take the second. Just my 2 cents. You really need to ask the resident gurus (e.g. blademaster) to be sure tho!


----------



## Piccolo71320

I currently have 2 BIOS chips
With a 0503 and a 5701 with

For 0503, I do not want to sacrifice and with the 5701 I can not until 3506 down.

The problem with the 5701 is that I do not change the 2nd Xeon on Gen2 can because after resetting the Bios these options do not appear.



As you can see from the pictures, the options are hidden even though the 2nd Xeon is installed.

German text:


Spoiler: Warning: Spoiler!



Ich habe aktuell 2 BIOS-Chips:
Einer mit 0503 und einer mit 5701

Denn 0503 will ich nicht opfern und mit dem 5701 kann ich nicht mehr bis zum 3506 runter.

Das Problem beim 5701 ist, das ich nach dem Zurücksetzten des Bios den 2. Xeon nicht auch auf Gen2 umstellen kann da diese Optionen nicht eingeblendet werden.

Wie man auf dem Foto sieht sind die Optionen ausgeblendet obwohl der 2. Xeon eingebaut ist.


----------



## squidman

Sorry mate a bit late here! But again I think blademaster and ors can prbly help you better. Still you will need to provide:

Chips and stepping and version (for
mobo

Mobo bios version

What you are running on other pci-e slots

VBIOS version


----------



## Piccolo71320

Quote:


> Originally Posted by *squidman*
> 
> Sorry mate a bit late here! But again I think blademaster and ors can prbly help you better. Still you will need to provide:
> 
> Chips and stepping and version (for
> mobo
> 
> Mobo bios version
> 
> What you are running on other pci-e slots
> 
> VBIOS version


I hope I understand the translation by Google right:

Xeon E5-2687W ES B0 stepping QA92

Main board revision 1.04

Bios:
0503 stable with two Xeons
5701 stable with a Xeon

PCI-E cards currently no installed.
If I find a BIOS that supports both Xeons, then a GTX780 is installed.

VBIOS 0.98 (onboard)

German text:


Spoiler: Warning: Spoiler!



Ich hoffe ich verstehe die Übersetzung von Google richtig:

Xeon E5-2687W ES B0-Stepping QA92

Mainbord Revision 1.04

Bios:
0503 mit beiden Xeons stabil
5701 mit einem Xeon stabil

PCI-E Karten aktuell keine verbaut.
Wenn ich ein Bios finde das beide Xeons unterstützt, dann wird eine GTX780 eingebaut.

VBios 0.98 (Onboard)


----------



## squidman

Dies ist ein Fall für Blademaster und die anderen Gurus Ich denke ...good luck!


----------



## Edgeloop

Hi everyone,
I came across this site while trying to find similar issues to mine online. So, thought I'd subscribe here and hoping someone can help me out.

I'm kinda desperate at the moment due to the fact that i have tried most troubleshooting without any success.

My current specs:

Asus Z9PE-D8 WS
2x Intel Xeon processors E5-2620 2.0Ghz
8x 4gb Dimms Corsair Vengeance
2x Intel BXRTS2011LC Liquid-cooled for socket LGA 2011
Corsair AX850 psu
Nvidia Quadro 4000
Crucial 256Gb SSD 
2Tb Hard Disk

I know this isn't a great rig but it's built to the budget I had at that time (2yrs)

Ok my problem with my workstation is that it doesn't want to boot up.
Every time I press the start button, the light goes on but then it turns off.

Only once in a while it would turn on but then again it would shut down after a few hours or so.

First I thought it was the socket as it started up in another room but now it just won't start up.

Anyone had this issue before?

Regards,

Kevin


----------



## Migonacci

Quote:


> Originally Posted by *squidman*
> 
> Looking at taobaoing same chips - e5-2695 v2's - (around 1000 yuan cheaper than e5-2697 v2's), what are you doing on say cinebench 11.5? Still can't get my head around cinebench r15 (getting old...like keeping up with 3d mark scores!)...unfortunately most of the 2695's seem to be low ghz...and I've only 'flea-bayed' so far not actually purchased from Taobao (do I REALLY? need a taobao agent?)..
> 
> 
> 
> 
> 
> 
> 
> 
> 
> I really need to see around 30-34 on Cinebench 11.5 when spending 2k usd taobaoing, otherwise it may be safer and more future-proof to just go the new v3 platform...


Hey Squidman,

Sorry for the super late response, my notifications for the forum didn't seem to be going through. I get a 2670 on cinebench 11.5. My current baseclock is at 105 mhz.

Also it seems nearly impossible or reliable to buy something from taobao....most of the prices for ES v2 processors are too high....and the sellers have zero room for negotiating. But I'm curious to know what the server market is like in China.....theres a crap load of xeons on that site......how will they manage to sell those ES processors?

Hope this helps.


----------



## squidman

Migonacci (italiano? forza gli azzuri!)

I haven't yet tried taobao (most 'helpful' websites recommend an agent which want 5-10?% of sales price in commission). Got my 2x e5-2665 v1 via ebay (which were prbly originally from taobao anyway...was a chinese reseller). Well done on the baseclock of 105! V3 ES xeons seem pretty cheap...prbly because they're trying to get rid of V3 ES stocks...and supplies of V2's are drying out. Almost tempted to just upgrade the mobo to a v3 compatible one, and use current mobo for a file server...but memory costs too! And what to do with my current xeons that noone wants! Not to mention migrating a raid5 array based off my motherboard!

Still, if it wasn't for my raid array, would prbly consider xeon v3 from taobao along with say a supermicro board (there's one with the pci-e slots space to accomodate double-width video cards...unlike the asus versions (where you lose a slot for every double-width vid card)...but yeah that upgrade is another 5-650 usd for the board, another 500? for ddr4 ram, then the ES chips themselves. So around 2.5k-3? USD as opposed to perhaps 10kyuan/1613 USD for a pair of E5-2697 v2's (going around 4-5 yuan each)....real dilemma...

My e5-2665's V1 with no bclk overclock, turbo up to around 2.4 ghz when all cores are running, and get around 17.8 on cinebench 11.5...which is a bit too slow once I start doing vray renders...about the same or slightly higher than an oc'd consumer i7 5960x? A pair of v2 e5-2697v2's get around 29-32 on cinebench 11.5, whereas the v3's are getting much higher scores...from 36-42 and up (iff you can afford the 18 core/36 thread version!)...

As a student, the 16 and 18 core versions are out of my ballpark, even the ES taobao versions...but the 12-14 core versions are doable (still around 5+yuan each!)...


----------



## squidman

Quote:


> Originally Posted by *Edgeloop*
> 
> Hi everyone,
> I came across this site while trying to find similar issues to mine online. So, thought I'd subscribe here and hoping someone can help me out.
> 
> I'm kinda desperate at the moment due to the fact that i have tried most troubleshooting without any success.
> 
> My current specs:
> 
> Asus Z9PE-D8 WS
> 2x Intel Xeon processors E5-2620 2.0Ghz
> 8x 4gb Dimms Corsair Vengeance
> 2x Intel BXRTS2011LC Liquid-cooled for socket LGA 2011
> Corsair AX850 psu
> Nvidia Quadro 4000
> Crucial 256Gb SSD
> 2Tb Hard Disk
> 
> I know this isn't a great rig but it's built to the budget I had at that time (2yrs)
> 
> Ok my problem with my workstation is that it doesn't want to boot up.
> Every time I press the start button, the light goes on but then it turns off.
> 
> Only once in a while it would turn on but then again it would shut down after a few hours or so.
> 
> First I thought it was the socket as it started up in another room but now it just won't start up.
> 
> Anyone had this issue before?
> 
> Regards,
> 
> Kevin


Kev, have you double checked the connections from your on/off switch to the mobo? I recall them being a bit flimsy, and not least asus documentation was a bit lacking there.. .guess I got lucky that it worked at all. Other thoughts (gurus should prbly chime in here)...perhaps the power supply? I have an ax1200 (and that generation of corsair PS are pretty well regarded)...would have thought you had more than enough juice tho! Other possible problems:

1) Cooling to cpu seated too tightly...my 2x corsair h80i's didn't like being screwed in too tight...had to loosen them a bit...had intermittent boot problems at first build until I loosened them a wee bit.
2) Try BIOS reset, with jumpers, and as last resort, removing the battery and reseating....
3) I had a lot of boot problems due to memory not being seated properly...it really needs to be in there!
4) It STILL happens to me...boot problems..sometimes it actually helps to turn off, but also unplug ps and just leave it for 10 minutes with no power into the system...then replugging all connections and trying again
5) guess you've already ruled out the local circuits (moving pc etc?)...
6) You're on the correct BIOS for your version of chips? Blademaster made a list of compatible ES chips and BIOS...
7) If a new build, you're sure that standoffs, etc between mobo and mobo-tray are correct and you're not shorting something out?

Feel your frustration! Took me ages to get a somewhat stable build....









Edit: have your problems only surfaced recently, and is your rig 2 years old? Did probs occur just after moving to a new room? I have yet to draw more than 450 watts from the wall, with like 15 hdd, both e5-2665's running balls to the wall on bclk of 105 (now reverted to stock), or the time I tried mining litecoins with my gtx 770 oc'd to breaking point...

Note in Scandinavia or UK and Europe, think we have different amperage limits from the wall as opposed to if you are say in 110V land....


----------



## squidman

Hi apropos my last posts...anyone with a software/fake raid off the mobo who also is thinking of upgrading to e.g. Asus or Supermicro mobos which support Xeon v3's who are in the same dilemma? How the hell are we going to do that? I have 10+ TB in raid5 off the mobo..any suggestions? Most of that is used too, have about 1tb free...and only 50 down/10 up in internet for the online backup...damn...









I guess one solution would be the seagate archival hdd's of 6-8tb, copy raid array over there...but have had such bad bad experiences with seagate....suggestions? (yeah a norco 24 bay is looking very tempting iff I could import one into Norway without 500 bucks in shipping + vat!)..

Solution 2 (may be simpler): sell flat, move from Bergen to Stavanger or Oslo or somewhere with fiber internet! Pretty laughable, but my folks live in Aus, with internet caps! You can't even use netflix or use online backup there! So I guess things could be worse (wow I feel really bad for aussies! or Canucks!)...can you even imagine caps? Maybe I should just move to Korea or Japan while I'm at it (great food, great internet, great powder skiing)...


----------



## squidman

Just an update..after reading about the bclk of 105, I just HAD to try that of course! No boot of course, suffice to say i'm now in hour 3 of trying to get it to boot (while I was at it, had to try and move my gtx 770 from slot 4 (black x8 pcie) to one of the blue ones, and move my sound card as well...still stuck with error codes 64-62 (pcx initialization) despite numerous cmos resets (damn that little jumper is a pain in the ass to find! even with my garish led pimping!).,...so guess lesson learned, if it ain't broke don't fix it! Damn just a shame can't get cards to work in the blue x16 pci-e slots....this board always was a pain in the arse to work with, will go with the supermicro next build....

http://www.servethehome.com/Server-detail/supermicro-x10drg-q-review-gpu-compute-server-motherboard/

!!


----------



## squidman

OK another update..if it ain't broke don't fix it! However after 7 hrs, successfully moved my audio card (asus phoebus) and vid card (gtx 770) to other slots, after several hours of bios resets with the godamn jumper.(which I can barely see! also a couple of battery reseats)..so now my 770 works (knock on wood!) for the first time on slot 3 (blue x16 pcie slot), and have made room for another card on top. Wow that is not recommended however...what a b!tch!....so purchasing supermicro after this!

Frankly no idea what you would do if you wanted to put say 3-4 watercooled vid cards in there for ray-tracing, scientific cuda-enabled apps or what have you (and you would have to do the dremel to the upper dvi to make it a single-slot anyway!)...chrissakes....this mobo really is a piece of work as far as frustration factor goes! And of course no way am I going to try oc the bclk again!


----------



## Migonacci

Quote:


> Originally Posted by *squidman*
> 
> Migonacci (italiano? forza gli azzuri!)
> 
> I haven't yet tried taobao (most 'helpful' websites recommend an agent which want 5-10?% of sales price in commission). Got my 2x e5-2665 v1 via ebay (which were prbly originally from taobao anyway...was a chinese reseller). Well done on the baseclock of 105! V3 ES xeons seem pretty cheap...prbly because they're trying to get rid of V3 ES stocks...and supplies of V2's are drying out. Almost tempted to just upgrade the mobo to a v3 compatible one, and use current mobo for a file server...but memory costs too! And what to do with my current xeons that noone wants! Not to mention migrating a raid5 array based off my motherboard!
> 
> Still, if it wasn't for my raid array, would prbly consider xeon v3 from taobao along with say a supermicro board (there's one with the pci-e slots space to accomodate double-width video cards...unlike the asus versions (where you lose a slot for every double-width vid card)...but yeah that upgrade is another 5-650 usd for the board, another 500? for ddr4 ram, then the ES chips themselves. So around 2.5k-3? USD as opposed to perhaps 10kyuan/1613 USD for a pair of E5-2697 v2's (going around 4-5 yuan each)....real dilemma...
> 
> My e5-2665's V1 with no bclk overclock, turbo up to around 2.4 ghz when all cores are running, and get around 17.8 on cinebench 11.5...which is a bit too slow once I start doing vray renders...about the same or slightly higher than an oc'd consumer i7 5960x? A pair of v2 e5-2697v2's get around 29-32 on cinebench 11.5, whereas the v3's are getting much higher scores...from 36-42 and up (iff you can afford the 18 core/36 thread version!)...
> 
> As a student, the 16 and 18 core versions are out of my ballpark, even the ES taobao versions...but the 12-14 core versions are doable (still around 5+yuan each!)...


Squidman,

Lol, no not Italian... just a screename I made up from "Fibonacci"

The upgrade to the v3 chips has a bit of overhead as you mentioned. New mobo, ram + cpus.....You're right about some of the 14 core v3s ...I see a couple of matched cpu auctions for ES's. It would be nice though to find a 18 core v3 chip ES at a reasonable price....monster parallel processing performance.

Fyi I'm also I'm looking to sell my e5-2670's v1 C0 stepping ES chips....in case you're interested.


----------



## Migonacci

Quote:


> Originally Posted by *squidman*
> 
> OK another update..if it ain't broke don't fix it! However after 7 hrs, successfully moved my audio card (asus phoebus) and vid card (gtx 770) to other slots, after several hours of bios resets with the godamn jumper.(which I can barely see! also a couple of battery reseats)..so now my 770 works (knock on wood!) for the first time on slot 3 (blue x16 pcie slot), and have made room for another card on top. Wow that is not recommended however...what a b!tch!....so purchasing supermicro after this!
> 
> Frankly no idea what you would do if you wanted to put say 3-4 watercooled vid cards in there for ray-tracing, scientific cuda-enabled apps or what have you (and you would have to do the dremel to the upper dvi to make it a single-slot anyway!)...chrissakes....this mobo really is a piece of work as far as frustration factor goes! And of course no way am I going to try oc the bclk again!


Squidman,

The small OC from upping the bclk is not really consistent across the board. It seems like some of the retail versions let you up the bclk pretty high. For my machine I topped out at 107....anything after that I had to reset the bios by pulling the battery.

see this vid from linustechtips


----------



## squidman

Got distracted by a thread on head-fi.org...but it's working again! phew! But my mobo wouldn't do bclk of 105...and it didn't like switching pcie slots of my 3 cards (770gtx, phoebus and raid card)..but seems to be stable now with 770 on slot 3...hopefully! Nightmare! Ok have to do some vid work here (apply LUT's to 50 gb of gopro footage...)..will leave you with some righteous DC GoGo...

(I had to pull the battery several times tonight just switching pcie slots!) not to mention innumereable messing with bios reset! Horrible...)..






enjoy! (sort of a DC thing, but as long as you can groove, well hey!)...and well despite 8+ years studying finance...never really liked math! Blech...music was more my thing!


----------



## Piccolo71320

Hello Squidman

Thanks for the tip with the 3506 bios> running!









Also, my GTX 780 runs smoothly in the blue PCI-E slot number 3.
Only the latest Nvidia driver 347.88 is not quite stable> now have the 347.25 in use and is stable.

Gruss Piccolo71320

German text:


Spoiler: Warning: Spoiler!



Hallo squidman

Vielen Dank für den Tipp mit dem 3506 Bios > läuft!

Auch meine GTX 780 läuft in dem blauen PCI-E-Steckplatz Nummer 3 problemlos.
Einzig der aktuelle Nvidia-Treiber 347.88 läuft nicht stabil > habe jetzt den 347.25 in Verwendung und der ist stabil.

Gruss Piccolo71320


----------



## Migonacci

Hello all,

I'm looking to sell my 2x E5-2670s (2.6GHZ) QB7A ES C0 stepping. If anyone is interested please PM me.

Thanks.


----------



## squidman

Keine Problem Gruss Piccolo!








So have my gtx 770 running on pci-e #3 finally, but requires numerous restarts (error code 94, 62 etc enumerating pci bus)...helps if I turn off computer, turn off powersupply at switch, drain power from all circuits, leave for 10 minutes!!! When I turn the power supply back on, the whole thing starts by itself...and 50% of the time it actually makes post.! Oh well, at least I freed up space for another card!


----------



## Syzygy1290

Does anyone know the amperage that I can pull through the fan headers? I have ap182 that can pull up to 1.3A.


----------



## squidman

Dunno...off the mobo fan headers? Using the old and extremely broken Corsair link...you're linking to one fan? (http://www.silverstonetek.com/product.php?pid=344)...

As an aside for those struggling with internal usb headers...wow I solved that with NZXT intenal usb expansion...http://www.nzxt.com/product/detail/99-iu01-usb-expansion-case-accessory.html...

Well solved that internal usb expansion problem when installing corsair h80i's for me!

Aaand iff you're still wondering..does Corsair link still suck balls? Yes it it does...(Is it still completely broken?) yeah pretty much...more the things change, more they stay the same...

(Back in the day, me and x thousand purchasers of the original corsair link went on a friggin' crusade!!!! against corsair for putting out such shoddy firmware/software!). There was blood, toil, sweat and tears, and generally lots of anti-Corsair sentiment on the floor after all was said and done...
!!


----------



## Syzygy1290

Yup, those are the ones. I have two in a ft04 and want to run them directly to the z9pe-d8 for software control. I just don't want to pull too much power through the onboard fan headers. Anyone used a greater than 1amp fan on this board (deltas, san ace, etc)?


----------



## squidman

Quote:


> Originally Posted by *Syzygy1290*
> 
> Yup, those are the ones. I have two in a ft04 and want to run them directly to the z9pe-d8 for software control. I just don't want to pull too much power through the onboard fan headers. Anyone used a greater than 1amp fan on this board (deltas, san ace, etc)?


You using a fan controller at the moment? Frankly no idea, never even touched internal mobo fan controllers (was my first build), just went dogshidt corsair from the get-go..(well iff the software worked..it would be friggin awesome! etc )...

I'm on a Corsair 900d (well at least they make sexy expensive cases!) and pretty much stock fans apart from 2x 120mm Noctuas. If I win the lottery, wow all those corsair POS fans are gone, replaced with noctuas...and new solid panels with sound-dampening foam everywhere, rubber grommets etc! On the other hand I would miss my led pimping!


----------



## Jeffinslaw

Hey guys, finally got some CPUs in to test my board that has been sitting in its box for nearly a year. Got two Xeon E5 2660s, one with stepping QB7B and the other with QA8X. Both 2660s so they should work together.

Well, I am getting error code 25 on the Q-CODE and there isn't an error 25 in the manual







any ideas? No video out either. Tried two different GPUs.

-Jeffinslaw


----------



## Jann

Quote:


> Originally Posted by *Jeffinslaw*
> 
> Hey guys, finally got some CPUs in to test my board that has been sitting in its box for nearly a year. Got two Xeon E5 2660s, one with stepping QB7B and the other with QA8X. Both 2660s so they should work together.
> 
> Well, I am getting error code 25 on the Q-CODE and there isn't an error 25 in the manual
> 
> 
> 
> 
> 
> 
> 
> any ideas? No video out either. Tried two different GPUs.
> 
> -Jeffinslaw


Don't combine different engineering sample CPU's (they both are 2660, but specs differ, as one is older).
Try booting up with just one cpu to see if they both work, but get a matching pair for dual cpu use.
I have dual QB7B, and they work fine.


----------



## Jeffinslaw

Quote:


> Originally Posted by *Jann*
> 
> Quote:
> 
> 
> 
> Originally Posted by *Jeffinslaw*
> 
> Hey guys, finally got some CPUs in to test my board that has been sitting in its box for nearly a year. Got two Xeon E5 2660s, one with stepping QB7B and the other with QA8X. Both 2660s so they should work together.
> 
> Well, I am getting error code 25 on the Q-CODE and there isn't an error 25 in the manual
> 
> 
> 
> 
> 
> 
> 
> any ideas? No video out either. Tried two different GPUs.
> 
> -Jeffinslaw
> 
> 
> 
> Don't combine different engineering sample CPU's (they both are 2660, but specs differ, as one is older).
> Try booting up with just one cpu to see if they both work, but get a matching pair for dual cpu use.
> I have dual QB7B, and they work fine.
Click to expand...

Well then why would I get the same error code testing the CPUs individually? Q-CODE goes 19, then 25 and just hangs there. With both CPUs tested individually and together.

As far as different stepping goes, I have only found one article (Tomshardware) about different CPU steppings and it was about the socket 1366 Xeons. I haven't found anything that says you can't use different steppings for the Sandy Bridge CPUs. It's just _more preferred_ to have matching steppings.

-Jeffinslaw

*EDIT* It turns out CPU 1 was borked. CPU 2 posted just fine.


----------



## Jann

Quote:


> Originally Posted by *Jeffinslaw*
> 
> As far as different stepping goes, I have only found one article (Tomshardware) about different CPU steppings and it was about the socket 1366 Xeons. I haven't found anything that says you can't use different steppings for the Sandy Bridge CPUs. It's just _more preferred_ to have matching steppings.
> 
> -Jeffinslaw
> 
> *EDIT* It turns out CPU 1 was borked. CPU 2 posted just fine.


I think steppings don't matter only with retail chips. But these ES are just too different. A buddy has two of those earlier 2660 ES chips, and they clock at 2.3GHz instead of 2.2 by default, so that's why I doubted it would work with the QB7B (which is a C0 revision or as close as it gets to retail).
Also, QB7B work with almost all BIOS versions (haven't flashed two latest versions), but QA8X probably requires an older BIOS to work at all.
Glad to hear not all is bad with your setup, and hope it all works out!


----------



## ironcurtain

Well, something after the latest BIOS upgrade disabled sleep for me. All ACPI settings look sane... bummer. Is there any guide for each BIOS setting also?
And when is ASUS going to give us up to date drivers for AHCI/Intel C600 storage? I'm even thinking of installing drivers unofficially 'hacked together'.
It's insane a high end board like this gets old drivers for its storage chipset.


----------



## RushiMP

I have a pair of E5-2689 Retail that I was thinking about using as a workstation.

After all this time, is this board still buggy or have the BIOS updates sorted out most of the problems?

If you had a choice to do it over again, would you consider a Supermicro X9DAI instead?


----------



## celemine1gig

Just a small update from my side:
If you remember, I had 2 Z9PE-D8 WS boards here. One with horrible RAM problems (slots/sticks disappearing or found to have "0 MB") and one with instability while using all 4 RAM slots per CPU.

The bad board was now replaced by Asus. At first they told me that the board was OK, because they obviously did not get the full problem description by the vendor and probably only tested with one RAM stick.
After I told the vendor (again!) what the detailed problem was and they told Asus, I finally got a completely new board back.
And this one is now working as it should. Running Passmark MemTest V6.0.0 Free right now, with 10346MB/s and no errors. :-D
In comparison the other boards (both) would give me about roughly 600MB/s and lots of ECC errors with the same RAM, CPU and PSU, when using all 4 RAM slots per CPU.

The interesting part is:
The hardware revision of the new board is still the same Rev 1.04, like with both the older boards. This means that my second board either has a hardware problem, like the really bad one, or Asus changed something else. Because I still see difference in behaviour concerning RAM.

I am currently looking into the CPLD as a possible source for the RAM stability fix. Could very well be that the hardware revisions are the same, but the software inside the CPLD is different. I got a programmer and will see if changing the CPLD's contents can make my second board play nice, too.
Will keep you updated. ;-)


----------



## lucek83

I've got few questions and it would be great if someone could help me out. I plan to build low end server:

Z9PE-D8C
E5-2620
1x16GB RAM KINGSTON 16GB DDR3 1600 ECCR (KVR16R11D4/16I)
I plan to expand it in future to 2x E5-2620 and 128GB of RAM.

The server will not be using any additional video card and no GPU related work will be done on it. It won't be mission critical unit so I don't require 99.9999% uptime.

My question is what PSU do I need? Do I need EPS or ATX? Will ATX 650W PSU be enough (again, this will be server, no additional video card will be used). Will such PSU at least handle my initial setup?

Thanks,
Lucek


----------



## ikem

I run
Quote:


> Originally Posted by *lucek83*
> 
> I've got few questions and it would be great if someone could help me out. I plan to build low end server:
> 
> Z9PE-D8C
> E5-2620
> 1x16GB RAM KINGSTON 16GB DDR3 1600 ECCR (KVR16R11D4/16I)
> I plan to expand it in future to 2x E5-2620 and 128GB of RAM.
> 
> The server will not be using any additional video card and no GPU related work will be done on it. It won't be mission critical unit so I don't require 99.9999% uptime.
> 
> My question is what PSU do I need? Do I need EPS or ATX? Will ATX 650W PSU be enough (again, this will be server, no additional video card will be used). Will such PSU at least handle my initial setup?
> 
> Thanks,
> Lucek


I run dual 2650s and a 290x, and while folding it pulls 450w from the wall. You will be good with a quality 650w.


----------



## celemine1gig

Quote:


> Originally Posted by *celemine1gig*
> 
> Just a small update from my side:
> If you remember, I had 2 Z9PE-D8 WS boards here. One with horrible RAM problems (slots/sticks disappearing or found to have "0 MB") and one with instability while using all 4 RAM slots per CPU.
> 
> The bad board was now replaced by Asus. At first they told me that the board was OK, because they obviously did not get the full problem description by the vendor and probably only tested with one RAM stick.
> After I told the vendor (again!) what the detailed problem was and they told Asus, I finally got a completely new board back.
> And this one is now working as it should. Running Passmark MemTest V6.0.0 Free right now, with 10346MB/s and no errors. :-D
> In comparison the other boards (both) would give me about roughly 600MB/s and lots of ECC errors with the same RAM, CPU and PSU, when using all 4 RAM slots per CPU.
> 
> The interesting part is:
> The hardware revision of the new board is still the same Rev 1.04, like with both the older boards. This means that my second board either has a hardware problem, like the really bad one, or Asus changed something else. Because I still see difference in behaviour concerning RAM.
> 
> I am currently looking into the CPLD as a possible source for the RAM stability fix. Could very well be that the hardware revisions are the same, but the software inside the CPLD is different. I got a programmer and will see if changing the CPLD's contents can make my second board play nice, too.
> Will keep you updated. ;-)


I have news.
It is somewhat embarrassing, however this was hard to catch.
My systems are now fine.
The RAM ECC error problem was caused by one of my ES Sandybridge CPUs. Due to the fact that I bought the CPUs indirectly in Asia, I have no chance for replacement. The CPU has a scratched/damaged gold contact. This damaged contact seems to be responsible for problems with RAM Slot number 3. RAM in slots 1,2 and 4 and everything works perfectly.









I can't even remember how much time I wasted on this issue.








For anyone else who is facing such problems, REALLY CAREFULLY check the CPU. The damage on my chip does not look that bad, but is has the said negative effects.

To sum it all up:
- One of my Z9PE-D8s really had a hardware problem (vanishing, varying RAM capacity/slots) --> RMA and exchange and everything is fine.
- One of my ES CPUs has a problem with the LGA gold pads on the CPU. This causes 1-Bit ECC Errors and really slow RAM speed, when RAM slot #3 is populated.

Now I can finally build my 2 Systems!








And then I will have some parts to sell, that I only bought for testing purposes, due to the various problems, that I was facing.


----------



## ironcurtain

Hm, anyone around can help with the sleep issue I described in my earlier post? I still can't use anything but hibernate in Win 8.1 x64. I will try resetting the BIOS once again, since my Nvidia drivers are up to date and everything seems fine.

Also, did Asus leave us all behind? When are they going to provide updated drivers for the C600 chipset?? Is this how they support a high end mobo? I own a EVGA SR-2 and the damn thing is still totally supported by EVGA.


----------



## squidman

New Issue for me...

I would like to upgrade to e5-2697 v2, but the newer v3 chips are cheaper on Taobao...by quite a lot. Guess they're flooded with v3 ES and QS chips at the moment. So it looks like the inevitable upgrade to v3 (hopefully with this supermicro mobo)...

X10DRG-Q (double space pci-e slots! yay! hard to fit pci spacers in my 900d case, would have to go the milkcrate route a la miners)...

However a big downside (apart from the 550-650 usd for the mobo, and xtra for ddr-4 ram), is how I'm going to migrate my raid 5 array now running off the intel sata/scu....any thoughts? I'm assuming it's basically hybrid hardware/software raid (have 4x4tb wd red for about 10.3tb useable space, all for backup of the other 9 hdd's). A real quandry...I guess I could wait 6 mnths for it to backup to my unlimited cloud backup service @ 10 mbps up, and then download again...or purchase a pair of seagate archive 6 tb drives...but still is mucho problemo...

Any advice appreciated...one thing's for sure..next system will have a proper lsi SAS raid card and battery backup to ease migration.









Edit: not going to throw any more monies at this system, but at a puny 32gb ram not very good for AE which likes 2GB per thread, and many suggest 1/4 of total system ram dedicated to other apps to avoid disk thrashing. So I would need say quite a bit more than 64gb ram to optimally use my 32 threads...well ebay to the rescue as usual...how does 128gb ram for 500 bucks strike you? hehe....

http://www.ebay.com/itm/8-VENTURA-16GB-x-8-128GB-LOT-PC3-12800-DDR3-1600-REGI-ECC-MEMORY-D3-62NA104SV-11-/171781557246?pt=LH_DefaultDomain_0&hash=item27fefa8bfe


----------



## ikem

Quote:


> Originally Posted by *squidman*
> 
> New Issue for me...
> 
> I would like to upgrade to e5-2697 v2, but the newer v3 chips are cheaper on Taobao...by quite a lot. Guess they're flooded with v3 ES and QS chips at the moment. So it looks like the inevitable upgrade to v3 (hopefully with this supermicro mobo)...
> 
> X10DRG-Q (double space pci-e slots! yay! hard to fit pci spacers in my 900d case, would have to go the milkcrate route a la miners)...
> 
> However a big downside (apart from the 550-650 usd for the mobo, and xtra for ddr-4 ram), is how I'm going to migrate my raid 5 array now running off the intel sata/scu....any thoughts? I'm assuming it's basically hybrid hardware/software raid (have 4x4tb wd red for about 10.3tb useable space, all for backup of the other 9 hdd's). A real quandry...I guess I could wait 6 mnths for it to backup to my unlimited cloud backup service @ 10 mbps up, and then download again...or purchase a pair of seagate archive 6 tb drives...but still is mucho problemo...
> 
> Any advice appreciated...one thing's for sure..next system will have a proper lsi SAS raid card and battery backup to ease migration.


why not get a Z10pe-D8?


----------



## squidman

No double space pcie...and almost all proper vid cards these days are double spaced unless you take a hack-saw to the top dvi output...









willing to tap my mobo tray for standoffs...don't want the milkcrate solution with pci extensions....the supermicro is circa 13x15" while the ssi-eeb of the asus boards is roughly eatx at 12x13"

Oh yeah...reliability...and 5? double spaced pci-e slots all at 16 or 8? Maybe gpgpu rendering will finally come of age....I do a lot of rendering...

and z10pe-d16 looks more tempting in any case than the d8...but the supermicro is a monster! Unless you go E7 and 4 cpu's (not too many taobao's of them yet, and not sure if the board will run with just one or two...)...

Of course I could just build a helmer render farm I guess..but then my beast wouldn't be that k!ick-ass..









In fact already almost out of pci-e slots..have my gpu, soundcard, and hba card in the bottom slots, so couldn't fit say a sli setup in there any more...(leaving my 770
gtx for phys-x and cuda rendering)...also almost out of spaces for hdd's (900d wasn't big enough!)...need to plop a 5-bay hdd backplane solution in my top 3x 5.25" bays soon...my raid 5 array is pretty much full...next step will be a norco 24-bay enclosure, but trying to put it off (impossible to buy in europe, and very noisy too).


----------



## squidman

"Also, did Asus leave us all behind? When are they going to provide updated drivers for the C600 chipset?? Is this how they support a high end mobo? I own a EVGA SR-2 and the damn thing is still totally supported by EVGA."

Thinking of migrating to supermicro...z9pe-d8 was difficult (especially for first build!)...albeit on ES chips


----------



## ikem

am I the only one that painted their Z9? But it is finally in a better case.


----------



## LeMakisar

Hi,
once again some weirdness with my z9pe ... it suddenly went giving 00 qcode.
unmounted everything, did everything i could with no success, then I just sswithjed cpus and it's ... almost working ... as it gives a b1 qcode and only cpu1 is detected.
I'm going to try switching back cpus again. Could a cpu die ??? Never seen that before ....

Any suggestion is appreciated !

Thanks

Edit: just switched cpus again and back to qcode 00. Really, does a cpu failure exist ??


----------



## squidman

Quote:


> Originally Posted by *LeMakisar*
> 
> Hi,
> once again some weirdness with my z9pe ... it suddenly went giving 00 qcode.
> unmounted everything, did everything i could with no success, then I just sswithjed cpus and it's ... almost working ... as it gives a b1 qcode and only cpu1 is detected.
> I'm going to try switching back cpus again. Could a cpu die ??? Never seen that before ....
> 
> Any suggestion is appreciated !
> 
> Thanks
> 
> Edit: just switched cpus again and back to qcode 00. Really, does a cpu failure exist ??


No idea, but for me it helps sometimes turning off, then powering off to release any random electricities (yes I know that is not very scientific, but then we're talking the asus z9 and voodoo!), and then waiting say at least a couple of minutes...and then replugging ps cord back in and restarting. Failing that, the whole reset cmos thing....sometimes believe it or not, just letting the stupid thing sit and stare at me for say a couple of minutes while completely out of electricity, after I have drained the motherfu...., just works...don't ask me why.

My latest prob is constant nvidia drivers just crashing...prbly? due to windows update what do I know? Regardless of driver version...sigh....we need an asus voodoo queen or something...willing to sacrifice a rooster or say some useless asus parts...


----------



## Syzygy1290

Quote:


> Originally Posted by *ikem*
> 
> am I the only one that painted their Z9? But it is finally in a better case.


Any pictures of the cable management in the back? I'm thinking about getting a tj10 as well, but I'm a little worried about the cable management.

Edit: Looks gorgeous btw


----------



## squidman

Quote:


> Originally Posted by *Syzygy1290*
> 
> Any pictures of the cable management in the back? I'm thinking about getting a tj10 as well, but I'm a little worried about the cable management.
> 
> Edit: Looks gorgeous btw


Second that! but wow would love to see a corsair 1000d double width with space for say oh I dunno 40 hdd's or something (man keep running out of space here!)...Corsair link etc sucks etc ditto customer service...but gotta say do love my 900d...just needs to be twice as big. And no mountain mods or caselabs doesn't do it for me...for that kind of money will fabricate my own case!


----------



## squidman

So think I tried to post before, still no ideas on how to migrate a raid 5 from this mobo (z9pe-d8 ws) to newer v3 xeon mobo_ Do I need to purchase a couple of seagate archive drives, copy the stuff over or what..
NB 4x4tb red in raid 5 on the sata/scu ports for total of 10.3 tb...wow would love some help here....sorry boring subject but yes at some point most of us are going to have to migrate to the newer platform...

A little loving for the newb please?


----------



## ikem

Quote:


> Originally Posted by *Syzygy1290*
> 
> Any pictures of the cable management in the back? I'm thinking about getting a tj10 as well, but I'm a little worried about the cable management.
> 
> Edit: Looks gorgeous btw


Quote:


> Originally Posted by *squidman*
> 
> Second that! but wow would love to see a corsair 1000d double width with space for say oh I dunno 40 hdd's or something (man keep running out of space here!)...Corsair link etc sucks ba!!s etc ditto customer service...but gotta say do love my 900d...just needs to be twice as big. And no mountain mods or caselabs doesn't do it for me...for that kind of money will fabricate my own godda...case!


You just have to think out of the box


----------



## squidman

Quote:


> Originally Posted by *ikem*
> 
> You just have to think out of the box


Yeah that's what I'm talkin' about! And will have to go edit my cussin I see again...sigh...


----------



## Syzygy1290

Sorry if this have been covered elsewhere in the thread, but has anyone else had issues with memtest86? I've recently upgraded to a new memory kit of 32gb samsung registered ecc ram, and have been trying verify that they are working properly. I got the newest version of memtest86 and have tried using both the lastest bios version (4.3.7) and the UEFI version 6.0.0

Both seem to run flawlessly in single core mode (ie the default of 6.0.0 and the single core option on 4.3.7). If I run in round robin or sequential mode with either version, memtest86 locks up and crashes within 30ish second usually when it switches to the second core. If I run in parallel mode on v6.0.0, I get lots of ECC correctable errors (haven't tried parallel on 4.3.7 yet). I've never filled up all the ram slots before, so I'm not sure if this is a cpu IMC issue, ram issue, or something to do with the z9pe-d8 and memtest software not playing nice together.

Bios is version 5304 using e5-2680 cpus. I will probably try updating the bios next and see if that helps things. Still have to test each stick individually as well. Any other suggestions?


----------



## celemine1gig

I can only speak from my experience with this board, (and also with Samsung ECC Reg RAM) but here is what I found:

- In my case, ECC-errors were caused by a dodgy CPU, that had problems with the 4th RAM channel. Using 1-3 channels was fine, but the fourth would ALWAYS cause ECC-Errors, no matter what. Could be a defective stick, though, too.
- Test all sticks alone to be sure that you don't have a single defective stick that is ruining stability for you.
- I had to increase VSSA voltage a little to get a stable system, once I used more than two RAM sticks per CPU. And maybe a little more dimm voltage, too, depending on the RAM you are using.

A last thing that I noticed:
In parallel processing mode, Memtest86 6.00 hangs the timer while doing its tests. It sometimes takes upto a minute to see an update and thus a indicaton that the system is indeed not hanging!








That was probably a coding error, where the timer did not get the priority it needs to still receive updates, no matter what mode is chosen.


----------



## aardvark0

Has anyone used LR-Dimms successfully with the Z9PE?
I tried out hynix and samsung 32gb 1866 sticks and they would give ecc errors with more than 1 per channel installed.
Each individual stick tested fine, but no combination of 2 sticks would pass memtest even with voltage boosted, timings reduced, lowered all the way 1333 speed.
Full with 8x 16gb sticks, no errors.

Are there any additional tricks for higher BCLK overlclock with pci-e cards installed?
I could get 2x 2696v2 stable up to 114 with just nvidia gpus installed, but with any other pci-e card installed like an areca 1882 raid, max is 105 - posts up to 114, but higher than 105 crashes as windows loads.

.


----------



## ocvn

Quote:


> Originally Posted by *aardvark0*
> 
> Has anyone used LR-Dimms successfully with the Z9PE?
> I tried out hynix and samsung 32gb 1866 sticks and they would give ecc errors with more than 1 per channel installed.
> Each individual stick tested fine, but no combination of 2 sticks would pass memtest even with voltage boosted, timings reduced, lowered all the way 1333 speed.
> Full with 8x 16gb sticks, no errors.
> 
> Are there any additional tricks for higher BCLK overlclock with pci-e cards installed?
> I could get 2x 2696v2 stable up to 114 with just nvidia gpus installed, but with any other pci-e card installed like an areca 1882 raid, max is 105 - posts up to 114, but higher than 105 crashes as windows loads.
> 
> .


Other PCI-E device can use if you change the PCI-E Gen3 or auto to 1. I dont use nvidia so I dont know the board will change PCI-E gen according to speed or not but with my AMD card, when I increase bclk to 113, the PCI-Gen change to 1. 100 -102 gen 3, 102-107 gen 2 and 108-113 gen 1 fully stable.


----------



## 82ghost82

Hi all!
I have just installed a gtx titan black in slot 5.
I applied the patch to force gen3 and read the usefull advice from theblademaster for editing bios setting: however, can somebody explain what is in the bios:
- IOH 0 and IOH 1
- ports 2A, 3A (under IOH 0) and ports 1A, 2A and 3A (under IOH 1)?

If I don't want to force all the pcie slots to gen3 what value I have to modify (if possible)?

Thanks

PS: now I have enabled all link speeds to gen3 and it's ok, but I'd want to force only the slot I need.


----------



## ViperJPB

Hey Everyone. Used to read and post here regularly. Glad to see the board is still going. Love my rig. However it's time to consider some upgrades and I havn't been up on the latest and greatest regarding Bios versions, CPU versions, and GPU compatibility. I seem to recall a lot of people having various problems with certain combinations. Certain Bios versions having issues..etc..But I can't recall the specifics so forgive me if this has been covered.

Basically want to update the system to Win 10 when it comes out from Windows 7. Plus I plan to purchase 2 of the new GTX 980 TI cards to replace the three GTX 680 4GB cards I currently have. Also upgrade it to 128GB of ram. So I'm gonna have to update my Bios for sure as I am on 0703. Which Bios's are safe and I seem to recall there being a certain order? Update to this one before updating to that one etc.

My specs are as follows:

Mobo: Z9PE-D8
CPUS: Two Xeon E5 2687W V1 Revision: C2
Memory: 32GB 1866
Drives: 2x256GB Corsair SSDs (One boots for work the other for play) 1x1TB WD Black, 8x1TB WD Black in a 8TB Raid 5.
GPUs: 3xGTX 680 4GB
PSU: Enermax 1200 watt
Misc: Blackmagic Decklinke Extreme 3D broadcast card, ATTO R680 internal SAS Raid card.

Any advice or help greatly appreciated.


----------



## TheBlademaster01

Quote:


> Originally Posted by *82ghost82*
> 
> Hi all!
> I have just installed a gtx titan black in slot 5.
> I applied the patch to force gen3 and read the usefull advice from theblademaster for editing bios setting: however, can somebody explain what is in the bios:
> - IOH 0 and IOH 1
> - ports 2A, 3A (under IOH 0) and ports 1A, 2A and 3A (under IOH 1)?
> 
> If I don't want to force all the pcie slots to gen3 what value I have to modify (if possible)?
> 
> Thanks
> 
> PS: now I have enabled all link speeds to gen3 and it's ok, but I'd want to force only the slot I need.


IOH = Input Output Hub, these are your PCI-E controllers. IOH 0 being the 40-lanes wide PCI-E controller on CPU0 and IOH 1 the same but on CPU1.

Device 1, 2 and 3 are different collections of Root Ports that switch between physical PCI-E slots on your motherboard (you can program the bandwidth for your ports here).

If you want to force slot 5 to gen3 you will need to force Root Port 2a of Hub 1 to gen3. This will also run slot 7 in gen 3 however (there is no way to only run one slot in gen3).
Quote:


> Originally Posted by *ViperJPB*
> 
> Hey Everyone. Used to read and post here regularly. Glad to see the board is still going. Love my rig. However it's time to consider some upgrades and I havn't been up on the latest and greatest regarding Bios versions, CPU versions, and GPU compatibility. I seem to recall a lot of people having various problems with certain combinations. Certain Bios versions having issues..etc..But I can't recall the specifics so forgive me if this has been covered.
> 
> Basically want to update the system to Win 10 when it comes out from Windows 7. Plus I plan to purchase 2 of the new GTX 980 TI cards to replace the three GTX 680 4GB cards I currently have. Also upgrade it to 128GB of ram. So I'm gonna have to update my Bios for sure as I am on 0703. Which Bios's are safe and I seem to recall there being a certain order? Update to this one before updating to that one etc.
> 
> My specs are as follows:
> 
> Mobo: Z9PE-D8
> CPUS: Two Xeon E5 2687W V1 Revision: C2
> Memory: 32GB 1866
> Drives: 2x256GB Corsair SSDs (One boots for work the other for play) 1x1TB WD Black, 8x1TB WD Black in a 8TB Raid 5.
> GPUs: 3xGTX 680 4GB
> PSU: Enermax 1200 watt
> Misc: Blackmagic Decklinke Extreme 3D broadcast card, ATTO R680 internal SAS Raid card.
> 
> Any advice or help greatly appreciated.


That's weird. I didn't think Kepler worked with BIOS version 0703. The PCI-E 3.0 issue was fixed in rev 3109 IIRC. In any case, you can just run the latest BIOS version as you have C2 CPUs (that also have VT-d erratum fixed).


----------



## geort45

Hi!

I stumbler upon these ES options:

e5-2650L rev B05
e5-2650 rev B08
e5-2670 rev B04

Can anyone please confirm compatibility







!!!

Thanks a lot!

Sent from my iPhone using Tapatalk


----------



## geort45

by rev I meant stepping, I forgot the term

Sent from my iPhone using Tapatalk


----------



## geort45

I have an sr2 with a somewhat power hungry gt480 and 6 ramsticks, on 620W

Sent from my iPhone using Tapatalk


----------



## TheBlademaster01

I'm not sure I follow. Are you trying to run 2x S2011 CPUs on a dual S1366 motherboard?


----------



## geort45

Damn, that post about my SR2 was a reply to someone asking about power consumption.

Sent from my iPhone using Tapatalk


----------



## ViperJPB

Thx for the response! Helps set my mind at ease for the upgrade process.


----------



## geort45

Quote:


> Originally Posted by *driftertravel*
> 
> Hello all,
> I'm trying to build a relatively cheap workstation and have picked up a z9pe-d8 ws for this purpose. I've been reading this thread a little, and, massive as it is, I have gotten the impression that I run some risks if I don't set everything up just right. I've always built my PCs but have never attempted a workstation build, so I could use some help.
> 
> For reference, the hardware I'm trying to use goes like this:
> z9pe-d8 ws
> 2x e5 2687W v2, printed qe2t es 3.2ghz
> 8x 8gb Kingston KVR16E11/8 ddr3 1600mhz ecc ram (haven't purchased yet, let me know if there's a better option, I still have 8x8gb corsair vengeance 1600mhz non-ecc ram from my last build, and was kind of hoping there was a chance it would work... )
> 2x Samsung 850 EVO 1TB SSD (RAID0)
> 4x Seagate ST3000DM001 3TB Sata Hdd (2x RAID0, maybe 1xRAID10)
> 4x NVIDIA GTX 780ti 3GB Graphics
> Corsair AX1500i 1500watt PSU
> 
> So my questions are thus:
> 1. Does anything seem amiss in the hardware lineup?
> 2. Is it correct that I should flash to BIOS 3109 and then to 5103 (but perhaps not beyond that?)
> 3. What else should I be concerned about/any advice?
> 
> I'd love to be able to take advantage of the experience you all have with these setups, so please let me know what you think.


Relatively cheap???!


----------



## LeMakisar

Quote:


> Originally Posted by *squidman*
> 
> No idea, but for me it helps sometimes turning off, then powering off to release any random electricities (yes I know that is not very scientific, but then we're talking the asus z9 and voodoo!), and then waiting say at least a couple of minutes...and then replugging ps cord back in and restarting. Failing that, the whole reset cmos thing....sometimes believe it or not, just letting the stupid thing sit and stare at me for say a couple of minutes while completely out of electricity, after I have drained the motherfu...., just works...don't ask me why.
> 
> My latest prob is constant nvidia drivers just crashing...prbly? due to windows update what do I know? Regardless of driver version...sigh....we need an asus voodoo queen or something...willing to sacrifice a rooster or say some useless asus parts...


Thanks, it was really one of the cpu that died ... never saw that before :s
anyway, so many problems with this board, got rid of it and went back to a more regular rig.

Thumbs up to Intel for the cpu RMA which was sooooo easy and fast.


----------



## ViperJPB

Alright so I just got my first GTX 980Ti in. Thx to this forum sounds like with the cpus I have I can just update straight to the latest bios from 0703. As I understand it the PCIE-3.0 issues have been fixed along the way. Anything I need to know or enable to get full PCIE-3.0 support to this new card? Don't have to use the force PCIE-3.0 nvidia patch any longer I presume.


----------



## ironcurtain

Oh well, after the latest BIOS update and changing something basically made my Win8.1 x64 installation kick into a dreaded Automatic Repair loop. Something is breaking apart. We are still stuck with old chipset drivers.

Has anyone experienced this? I'm using Samsung 840 SSDs and ECC memory. Both have been tested and work fine. I used Memtest86+ and completed a batch of tests (with 64GB it took a while....).

I'm considering a clean reinstall in all honesty. I tried everything I could to no avail, from restoring the BCD, to trying to check the boot logs, which seem garbled (the text in them is binary data, not Unicode text).


----------



## ironcurtain

I'm having issues with the ASMB6-iKVM add-on.

It's working just fine when the system is powered but apparently when I try to power cycle or shut it down it basically dies. My PSU is a Corsair AX 1200 (not the "i" model) and I don't have a way to connect the SMBus socket (PSMBUSMB1?), since my PSU doesn't support it. I wonder if there is anything suitable that emulates it or if I can overcome the problem some other way, assuming this is the root cause of why my ASMB6 dies when the machine shuts down, instead of remaining active.

Any ideas? I'm traveling soon and need to leave this system accessible.

For now I setup a failsafe-kinda RTC based power on schedule in the BIOS, we will see if that works. I cannot remotely power cycle the system right now.


----------



## ignoramus

I've done something!! Help!

Rig:
Z9ped8 ws
2 x 2687w Qa92
Nvidia Gtx 460
64 gb ECC

Was working fine on bios 3506, as in 24/7 folding & rendering stable.

Decided to tighten up ram timings (out of boredom) and now it won't post. Q-code: 66

Went from 9-9-9-24 tRFC:107 to 8-8-8-22 tRFC: 88 and it booted fine, completed cinecench. Then tried 7-7-7-20 tRFC: 88 and it bluescreened and has failed to post since. Have I damaged my ram?

Tried resetting bios with the jumper and removing the little battery for 10 minutes.. Same Q-code:66

What do I do?!


----------



## ocvn

Quote:


> Originally Posted by *ignoramus*
> 
> I've done something!! Help!
> 
> Rig:
> Z9ped8 ws
> 2 x 2687w Qa92
> Nvidia Gtx 460
> 64 gb ECC
> 
> Was working fine on bios 3506, as in 24/7 folding & rendering stable.
> 
> Decided to tighten up ram timings (out of boredom) and now it won't post. Q-code: 66
> 
> Went from 9-9-9-24 tRFC:107 to 8-8-8-22 tRFC: 88 and it booted fine, completed cinecench. Then tried 7-7-7-20 tRFC: 88 and it bluescreened and has failed to post since. Have I damaged my ram?
> 
> Tried resetting bios with the jumper and removing the little battery for 10 minutes.. Same Q-code:66
> 
> What do I do?!


Take out all Rams and test with single stick for each cpus. If this doesnt work, Re-seat both cpus will reset everything.


----------



## ignoramus

Reseating the cpus resets things more than clearing the bios/removing the battery?


----------



## ocvn

Quote:


> Originally Posted by *ignoramus*
> 
> Reseating the cpus resets things more than clearing the bios/removing the battery?


Yeah. you can check back the thread, we discussed quite a lot abt reseat cpu


----------



## ignoramus

Getting nervous.. Nothing's working!

Have tried:
-Reseating cpus (actually swapped them)
-Booting with one stick of ram per cpu (a1 and e1)
-Using just 1 cpu with ram (the left one, filling all ram slots - it didn't power up at all

I think i got the Q-code wrong though. It might be two lower case B's as opposed to 66. If I try to boot with no ram, error code is b0

Help!


----------



## geort45

Quote:


> Originally Posted by *ignoramus*
> 
> Getting nervous.. Nothing's working!
> 
> Have tried:
> -Reseating cpus (actually swapped them)
> -Booting with one stick of ram per cpu (a1 and e1)
> -Using just 1 cpu with ram (the left one, filling all ram slots - it didn't power up at all
> 
> I think i got the Q-code wrong though. It might be two lower case B's as opposed to 66. If I try to boot with no ram, error code is b0
> 
> Help!


Try with RAM you HAVEN'T used in that Rig, maybe you fried them up


----------



## ignoramus

Thanks guys, I got it going. Turns out all the information related to the correct sockets was in the first post.. Sorry for the junk posts!

Tried 1 cpu and 1 stick of ram - this time in the correct place - and found that 1of8 sticks was able to get to post. The funny thing is the bios settings never reset despite clearing the bios and removing the battery. The same settings that the ram couldn't handle were still present. Aren't the settings supposed to reset to defaults when the bios is flushed??


----------



## jimjam47

Hi,

I'm new to this forum, so I apologize for any lapses in posting etiquette.

I need a little bit of help solving a problem with my ram install.

I am running 2 e5-2620 v2's with a 1300 watt psu and a GTX 740.

I recently upgraded to a 64 gb g.skill ripjaw z (2133) 8 module kit with the goal of running in quad channel. Previously I was running two 16 gb ecc sticks (which are now for sale if anyone is interested)

I installed the modules and the bios initially posted at a full 64 gb, but I noticed that in the bios it reported that DIMM C1 was present but with 0 mb. I have also noticed that at post it intermittently posts 56 gb. Inside the bios the DIMM configuration information reports that DIMM C1 is populated with 0 mb, this is consistent whether 64 gb or 56 gb are detected at post.

CPUZ reports 64 gb in triple channel consistently, Windows 7 reports that either 64 gb or 56 is present but consistently reports that 56 is usable.

Things I have tried:

1) updating the bios to most recent

2) unscrewing both cpu coolers, and booting with the cooler just resting on the cpu's not, latched in. I also tried loosening all of the screws holding the motherboard done, that did nothing.

3) switching all of the dimm/dimm socket locations, to rule out that it wasn't a bad dimm

4) using a 32 gb 8 module memory kit - same problem - Dimm C1 0 mb - Windows 7 reported 28 gb

from reading through the previous posts I am a little unsure what to try next. I'm going to try switching the CPU sockets, to see if it is CPU dependent, and also see what happens if I de-populate the 4th dimm slots for each cpu but I am looking for a whole batch of suggestions to try before I go through the hassle of pulling my workstation all apart again.

Thanks in advance.


----------



## KnownDragon

I will shortly be getting this board. So I have a quick couple of questions.

Heatsinks which fit and are the best? Everyone has an opinion and I will listen.

Overclocking Ivy bridge xeons 2600 series anyone attempted this?

Will windows 8.1 work with both cpu's or do I need Pro or Serv.

The system I am planning on building is for light serving and gaming.


----------



## Jeffinslaw

Quote:


> Originally Posted by *KnownDragon*
> 
> I will shortly be getting this board. So I have a quick couple of questions.
> 
> Heatsinks which fit and are the best? Everyone has an opinion and I will listen.
> 
> Overclocking Ivy bridge xeons 2600 series anyone attempted this?
> 
> Will windows 8.1 work with both cpu's or do I need Pro or Serv.
> 
> The system I am planning on building is for light serving and gaming.


Should have just PM'd me mate









1. The current heatsinks work just fine for me. Like in any case set up, just make sure you have airflow over the fins to keep them cooled.
2. No overclocking on ANY of the E5 2XXX or E5 4XXX Xeons. You can overclock the E5 16XX series but those CPUs can only be used in a 1P configuration. Google "quickpath interconnect" for more information on that.
3. Windows 8.1 will work just fine.

Hope I answered your questions









-Jeffinslaw


----------



## NapalmV5

http://rog.asus.com/rog-pro/realbench-v2-leaderboard/










http://www.overclock.net/t/1518806/fire-strike-ultra-top-30/1170#post_24186364


----------



## woops

Asus Z9PE-D8 WS and Windows 10 Sound

Hello, I can only get stereo. Any fixes available for 5.1?

Thanks for any assistance you can give!


----------



## Trekki

Hello,

anyone experimented the Dual 2630v2 overclocking, or similar of these CPU.

At this time, i got 110mhz without freeze and problem, do you think its possible to get a 125 or 130mhz without damaged component?
I put a little more Voltage for Vcore and VSA core (Vcore + 0.1) (VSA + 0.15)

Here my build :

MOBO = Z9PED8-WS (5103 bios)
PSU = EVGA Supernova 1000G2 1000W 80PLUS Gold
CPU(2x) = Intel® Xeon® Processor E5-2630 v2 (15M Cache, 2.60 GHz) (26 * 100mhz)
AirC(2x) = Noctua NH-U12S LGA2011/115X AM2+ AM3+ FM1 FM2
RAM(4x) = Kingston 16GB 240-PIN DDR3 ECC Registered DDR3 1600mhz
GPU = EVGA GTX 780 SC 3 gb
HardDrive = WD BLACK SERIES WD3003FZEX 3TB 64MB Hard Drive
HardDrive = Kingston SSDNow V300 Series 240G 2.5" 240 GB SATA III


Here an image of the 110mhz.

At this time,i have a 110mhz * 29 (with turbo at 31), this is possible to put that around 125 or 130mhz?

I tested 115 without freeze or bug, but i dont know how to fix my GPU, i dont want to OC more my GPU. And what is the best voltage for every thing in the box!

Anyone can help me?

Ty


----------



## phillydee

S
Quote:


> Originally Posted by *woops*
> 
> Asus Z9PE-D8 WS and Windows 10 Sound
> 
> Hello, I can only get stereo. Any fixes available for 5.1?
> 
> Thanks for any assistance you can give!


Hi! You're on Windows 10? Did you install the Chipset Drivers from Asus? How did your install go? Did you go from 8.1 to 10? Sorry I'm not of much help with audio; I am still on W8 and use a Sound Blaster PCI card for sound.


----------



## woops

phillydee,

Yeah, that's a great idea, just get an audio card. They are real cheap and will probably have drivers from windows 10 already included.

Thanks.


----------



## woops

For Windows 10,

Just install the Windows 8.1 MIO_888_892_ALC898 driver you can download from
the asus Z9PE-D8 WS page.

5.1 will work.


----------



## Blindsay

anyone install ESXi on this? trying to install 6.0 and I am getting fatal error 10 during the install

edit: gave up on 6.0 and got 5.5 installed but for some reason the local vsphere client is freezing when trying to connect to the host


----------



## RushiMP

So this thread has gotten very long and there are no links in the

So my Dell Precision T5600 was being testy and I broke down and bought one of these beasts. I have it loaded up with 64GB LP Crucial Ballistix ram and everything seems cozy.

Does anyone have experience with optimal BIOS settings for workstation / computational workloads?

Also, what might be the highest base clock that can reliably be achieved and are there any BIOS tweaks to allow for it. I have seen some people getting up past 110 and am very curious.

Thanks in advance.


----------



## wirlybird

Hi all. First time poster.

Finally breathing life into my Z9PE-D8 WS!! Running a bare bones config. MB, power supply, graphics card, memory and cpu's.

Mem = Crucial DDR3-1600 1.35v DR x4 RDIMM 240p 16GB X 4.

2.0 CPU's

Issue I am having is this. Posts with 1 cpu and 4 mem sticks on 1st cpu, also with 2 sticks on cpu. If I put the 2nd 2 sticks on the second cpu (still 1 cpu) it gets flakey on post. If I add the second cpu it doesn't post but just seems to cycle as if you keep pressing the reset. I am guessing incompatible cpu's.

Thanks for any ideas at this point. I am prepaired to buy proper cpu's if this does seem to be the issue!!

Me again!! Please do not hesitate to recommend a pair of cpu's that won't kill the bank!! I am not opposed to used at all!!


----------



## wirlybird

Me again!! Please do not hesitate to recommend a pair of cpu's that won't kill the bank!! I am not opposed to used at all!!


----------



## jimjam47

Just an update.

I RMA'd the board. When I disassembled it and re-seated the CPU's I noticed that there were bent pins in Socket 1.

If I recall, another member had a similiar problem as me, and it was likely related to bent pins.

The RMA process took less then two weeks and was completely trouble free. I was really impressed. I think they replaced my board with an open box one (communication was limited), and now all DIMM banks are populated and working.

The ram posts at 1600 with a total of 64 gb in quad channel (according to cpu-z).

My next question is: When I try to run this ram at 1866 or 2133 the system wont boot.

The ram is a G.Skill Ripjaws Z DDR3-2133 cl10-12-12-31 1.6v 64 gb kit. I have manually inputted the timing data, and adjusted the dram voltage. That works fine, but when I force 1866 or 2133 the system wont post.

Things are pretty zippy with the board working and all ram installed, but I'd like to push the ram if anyone has suggestions of settings to try. Is there a VSSA setting or something that I should try? I'm a newb at overclocking, so I need a little bit of guidance.

If I don't get any help on this forum I'll just leave it as is - but I think I've seen a few users on this forum running this same ram with a similar setup at 2133 mhz.

Thanks,

JJ


----------



## jimjam47

wirlybird, what cpu's are you using now?


----------



## RushiMP

I have the strangest problem.

Latest BIOS and Retail E5-2689 cpus. Very stable.

I can't disable hyperthreading. When I disable HT in the BIOS the computer will POST but will hang before loading the OS or a USB stick. It just sits there with a blinking cursor in the left upper screen. The problem goes away once I enable HT again. Anyone else seen this?


----------



## NapalmV5

any other z9 firing on all cylinders? lets see what them dual 6/8/10/12 core can do @

http://www.overclock.net/t/1556739/bmd-cpu-thread-benchmark-x64-v2-02-bug-fix-update-8-10-2015










http://rog.asus.com/rog-pro/realbench-v2-leaderboard/










http://www.3dmark.com/3dm/8182803


----------



## HiTekJeff

Quote:


> Originally Posted by *woops*
> 
> Asus Z9PE-D8 WS and Windows 10 Sound
> 
> Hello, I can only get stereo. Any fixes available for 5.1?
> 
> Thanks for any assistance you can give!


Mine work just fine with Windows 10, but you can always download the most current drivers direct from the RealTek web site. Look for the "RealTek HD Audio Drivers" and install. Otherwise, check the BIOS to make sure everything is enabled and set correct for your audio there.

Hope that work, good luck.


----------



## HiTekJeff

I started seeing a strange behavior when I unplug then plug back in the main power cable to my computer. It will instantly startup, I get it to POST just fine with no issues and have check the power switch and MOBO leads to make sure one didn't come off or was lose.

I am seeing a "Q" code 40 on the motherboard LED display. When checking the manual, I don't see a reference to a code 40, so I don't know what that means. Does anyone know or has see this before? It must have something to do with the power issue, but there's no way to know until I find out if those codes have changed or been updated.

Thanks for any help.


----------



## NapalmV5

Quote:


> Originally Posted by *RushiMP*
> 
> So this thread has gotten very long and there are no links in the
> 
> So my Dell Precision T5600 was being testy and I broke down and bought one of these beasts. I have it loaded up with *64GB LP Crucial Ballistix ram* and everything seems cozy.
> 
> Does anyone have experience with optimal BIOS settings for workstation / computational workloads?
> 
> Also, what might be the highest base clock that can reliably be achieved and are there any BIOS tweaks to allow for it. I have seen some people getting up past 110 and am very curious.
> 
> Thanks in advance.


non-ecc? then i got the same ram and a 32gb which helps z9 to 114 bus but that was on 2x gtx580 pcie2.0 and not 4x titanx pcie3.0

here are my 110 bus optimal bios settings 24/7 air cooled - thermalright copper 120s


----------



## RushiMP

Thanks alot, let me see if I can duplicate some of your success.


----------



## RushiMP

Well you helped me get past my 104 road block, but it would appear my CPUs require gobs of voltage past that to remain stable.

I wish I could force them to maintain their top all core turbo bin of 33X.

My first CPU fluctuates between 32-33X @ 55C and the second one wavers 31-32X at about 60C

I adjusted the short and long duration power limits to 150 and 200.

Are those temps really limiting them? Any other BIOS settings I can play with?


----------



## NapalmV5

edit2: leave short and long duration power limits at 0 hence the multiplier fluctuates.. the 0 settings are optimal

well thats the thing about z9 apart from the less than basic voltage settings not much to play with you play with gpus and power

on radeon 290x's 102 bus max at 103 video signal loss even though system keeps posting on

on gtx580s 114 max with no video signal loss

on titanx i can barely take it to 113 but this is on the 64gb kit.. will go back to the 32gb kit today

whats your gpu/s ?

hows +12v and +3.3v rails doing at bios ?

if you dont have gtx 580/titan/titanx its best to lower the pcie gen to 2 and even gen1

edit: just like i suspected what prevents higher bus on 4x titanx is the dual psu

this is on a single titanx and psu at 114 bus


----------



## RushiMP

As it is set now I average 31.5X on CPU 1 and 30.5X on CPU 2. Both are capable of running 33X.

I am going try and set the power limits back to 0 and see how it goes.

I plan on running Titan X on this board.


----------



## NapalmV5

same here when i set the values to anything other than 0 the multis fluctuates on both cpus even at idle

titan x is not going to be the bus limiting factor


----------



## RushiMP

Even with all power values at 0 I still get the same 1-1.5X lower top all core turbo then I expect.


----------



## NapalmV5

edit: your max ratio is 36x just like mine.. set it to 36x if its not at ai tweaker

so i downloaded hwinfo64 and at idle it fluctuates between 34 and 36 even drops to the min 12 all the while it remains at a steady 36x110 at cpuz which is what ive always went by.. though at full load it remains at 36.. referring to both cpus

hows yours at full load?


----------



## RushiMP

Yeah, still no joy. I can duplicate your settings but when running [email protected] on all the cores it runs 1-2X below max all core turbo no matter what settings I choose.


----------



## sonidosolidario

Hi!

This is my first build, So far so good, until:

I have 2 x gtx 780ti, each from a different brand.
The problem is that only one card its enabling CUDA, which it´s bad because the rig main goal was to do GPU rendering.
The OK card it´s on pcie 3, and the NO-NO card it´s on pcie 5.
This has something to do with the BIOS, or could be an issue of the drivers?

Thank you!

Ps:
Sorry for my bad english


----------



## renderbox

Quote:


> Originally Posted by *ViperJPB*
> Basically want to update the system to Win 10 when it comes out from Windows 7. Plus I plan to purchase 2 of the new *GTX 980 TI* cards to replace the three GTX 680 4GB cards I currently have. Also upgrade it to 128GB of ram. So I'm gonna have to update my Bios for sure as I am on 0703. Which Bios's are safe and I seem to recall there being a certain order? Update to this one before updating to that one etc.


Quote:


> Originally Posted by *TheBlademaster01*
> 
> That's weird. I didn't think Kepler worked with BIOS version 0703. *The PCI-E 3.0 issue was fixed in rev 3109 IIRC*. In any case, you can just run the latest BIOS version as you have C2 CPUs (that also have VT-d erratum fixed).


Hello everbody,

I am a proud - and so far happy - owner of the Z9PE-D8. But the above discussed combination of a Z9PE and 980ti graphis card doesn't work for me. I upgraded to the latest board BIOS 5701 (13.02.2015). But the booting process always stops (after approximately 23 seconds) at the "b2" state as discussed in this ASUS Z9PE-post two years ago.

Has anybody tried this combination successfully? Any hints or ideas what to do? Opinions and replies welcome









Regards
Matthias


----------



## squidman

Sorry new topic, couldn't find it in the thread. Migrated to win 10, and apart from a few hiccups, no problems until now (ok some major hiccups! But needed win10 for my new 4k monitor and the separate display scaling).

Just today, it appears 3! of my 4 WD RED 4 TB drives in a raid 5 array off the intel sata/scu controller have failed..or are failing in BIOS as well as the intel IRSTe software. However, both disk management and Paragon 12 hdd manager shows a complete (empty) volume. The IRSTe and BIOS however show 1 disk as working and 3 as failed. The only error I could find was that they hadn't 'migrated' successfully. Luckily, most of it is backed up to the cloud, however, no way of knowing what will have disappeared...such huge amounts of data (10 TB!) are kind of hard to keep up to date on a measly 10 Mbps up cable connection.

I had to install an intel driver (SATA/SCU c600 driver) which has been working for the last 3 weeks) as of course asus has no drivers for the z9pe-d8 ws (if they ever will).

On MB Bios 3506, and 2x ES B1 e5-2665...

Any ideas? Anyone running raid off those bottom 4 intel sata/sas ports who has made the migration to win 10? Am kind of desperate here...


----------



## GendoIkari

Hi all. I have either bios 5103 or 5104, and I'd like to upgrade it to the latest so hopefully I don't have to unplug all my USB stuff besides my keyboard and mouse at boot to be able to use the keyboard at boot. and in bios. I have also have four water cooled GTX 580's and two 640's, along with two raid sets, and I was wondering if the system would still boot after upgrade with all those cards installed, and if the bios upgrade would somehow affect my RAID sets, by upgrading or affecting the on-board LSI controller. Also, do I have to clear the existing bios settings before upgrading, for best results?


----------



## squidman

After my experience with raid 5 off the SCU/SAS/SATA ports, I would definately make a backup of the raid sets. At the moment, my 4 x 4tb wd red nas hdd's are showing as online (from 3 failed!) however the raid 5 volume is still showing as failed. Possibly it's taking a long time to migrate/and or initialize? Although with raid sets, I'm (and very likely you as well) looking at something like 8tb seagate archive drives just to secure the data (that is If I don't lose it all...think my last proper backup was a couple of months ago online). Just picked up another 2 wd red 4tb nas drives today, hoping I could rebuild my raid 5, but no dice so far...and of course there are only 4 damn sas/scu ports there.....at least they're not showing as failed for the moment in the IRSTe software or bios....

Man no more fake raid 5 off the mobo for me though, and even leery of doing it with say a lsi card with raid 6 and battery backup....


----------



## squidman

Well the RAID 5 off the 4 bottom 4 SCU/SATA/SAS ports died as expected....luckily I had online backup of most of it. Did mean another 2 WD Red drives to try and save them. Now running pure software raid in the form of windows 10 storage systems in basically raid 1+0. The downside with the hardware raid on our mobo (z9pe-d8) is that it's 0+1, which is just too risky! So I think never again raid 5...from now on raid 1+0 or maybe 6, and if with raid card, then no battery backup which dies after 3 days, but has to be LSI card with nand/memory-cache.

So now at what 6? 4tb WD red hdd's, and only getting 11 tb's out of it...and another month to download the data. Wow, lesson learned, don't ever raid off the mobo for sensitive data! Not sure iff I can quite swing the $$$ for the tape backups yet tho! No fiber for me yet unfortunately....but thank god no data quotas! Must suck living in Australia, Canada etc...boggles my tiny mind! Can't even do online backups!


----------



## linol

Any owner here who uses a pci-e adapter and M.2 ssd on a asus z9pe-d8 ws?









Any problems to get it to boot from the pcie/ssd?

What modell of adaptercard and ssd?


----------



## squidman

More troubles! with the old z9pe-d8 ws.....

Upgraded from 32gb RAM to 64, bought a 32 gb Corsair kit pretty much identical to the last (1866 cl10 but not red). Thing is, it looks like (after exhaustively swapping out modules, reseating for the nth time etc), the dimm slot to one of my cpu's is dead. It identifies on bios (DIMM G1) and shows there is a module there, but RAM shows as 0 no matter which ram dimm I use. So a bit bummed out. Tried cmos reset (not battery reset yet).

Any ideas? A couple of vid programs (FFMPEG, AE, Resolve, etc) really eat up the ram, so was hoping this would help before inevitable move to socket 2011-3. Mostly hoping throwing more RAM at Adobe would enable more cores being used at closer to 100%...

This mobo was purchased at a steep discount/customer return, so maybe I just found out why I picked it up for half price those 3? years ago...I only had 4 sticks of 8gb modules until now..

Thanks as per usual...


----------



## natapon

Intending to get this board (or any dual 2011 v1/2 board) for a pair of QE1X laying around (worked together before in another system). Don't see this stepping on the E5 Xeon list under Ivy Bridge-EP. Closest I see there is

Code:



Code:


2660 v2 ???  QDNS    R0      2.2     6/12    60      Need to verify model

Which matches the stepping R0.

Are the chances good for the pair to work on this board?

Cheers!


----------



## bigblock990

Need some help getting pcie 3.0 to work. I went into bios cpu IIO bridge config and changed all PORT Link Speed from auto to GEN3.

system specs:
bios 5601
dual Xeon E5-2643 SR0L7
lubuntu 14.04
evga gtx 970 in pciex16_1 slot
nvidia 346.96 driver

cpu's, mobo, and gpu are all pcie 3.0 devices, but can only get pcie 2.0



Edit:
Updated bios to 5701, redid bios settings, reinstalled nvidia drivers, still stuck at pcie 2.0


----------



## TheBlademaster01

Quote:


> Originally Posted by *bigblock990*
> 
> Need some help getting pcie 3.0 to work. I went into bios cpu IIO bridge config and changed all PORT Link Speed from auto to GEN3.
> 
> system specs:
> bios 5601
> dual Xeon E5-2643 SR0L7
> lubuntu 14.04
> evga gtx 970 in pciex16_1 slot
> nvidia 346.96 driver
> 
> cpu's, mobo, and gpu are all pcie 3.0 devices, but can only get pcie 2.0
> 
> Edit:
> Updated bios to 5701, redid bios settings, reinstalled nvidia drivers, still stuck at pcie 2.0


I never really looked into this and have the same problem in my Linux distros, but apparently you can edit nvidia.conf to enable Gen 3 in Linux. For Windows you need the pci-e 3.0 patch (which can be downloaded from TPU)

Linux:

https://devtalk.nvidia.com/default/topic/545186/enabling-pcie-3-0-with-nvreg_enablepciegen3-on-titan/

Windows:

http://www.techpowerup.com/168027/nvidia-releases-pci-express-gen-3-0-enabling-patch-for-sandy-bridge-e-hedt-platform.html

I'll try to look into it on my machine tonight.


----------



## bigblock990

Thanks, I will try this after work. I searched for a solution and didn't come across that thread.

It must be an issue with the z9pe mobo/chipset?? I have lubuntu 14.04 running on a z87 rig and pcie 3.0 works correctly without any workarounds.


----------



## TheBlademaster01

Quote:


> Originally Posted by *bigblock990*
> 
> Thanks, I will try this after work. I searched for a solution and didn't come across that thread.
> 
> It must be an issue with the z9pe mobo/chipset?? I have lubuntu 14.04 running on a z87 rig and pcie 3.0 works correctly without any workarounds.


It's a conflict between the PCI-E controller in Sandy Bridge-E/EP and nVidia firmware. I read a couple of years ago that at the time that Intel was doing validation for the Sandy Bridge HEDT platform they had no functional PCI-E 3.0 devices. You wouldn't have this problem if you popped in Ivy Bridge processors for example.

I explicitly named Linux because I knew from tech news that there existed a patch for Windows. I didn't know that there was a similar workaround for Linux.


----------



## bigblock990

Quote:


> Originally Posted by *TheBlademaster01*
> 
> It's a conflict between the PCI-E controller in Sandy Bridge-E/EP and nVidia firmware. I read a couple of years ago that at the time that Intel was doing validation for the Sandy Bridge HEDT platform they had no functional PCI-E 3.0 devices. You wouldn't have this problem if you popped in Ivy Bridge processors for example.
> 
> I exclusively named Linux because I knew from tech news that there existed a patch for Windows. I didn't know that there was a similar workaround for Linux.


Well I read through the devtalk thread and some of that went over my head as I'm pretty new to linux. So I will have to do some more google searching to figure out how to properly try some of those suggestions. Anyways, hopefully I can get it to work. Worst case would be to sell these two e5-2643 and pick up a couple e5-26xx V2.


----------



## bigblock990

Turns out it was super easy







Huge thanks to theblademaster01

All I did was open terminal,

Code:



Code:


sudo nano /etc/modprobe.d/nvidia.conf

type in

Code:



Code:


options nvidia NVreg_EnablePCIeGen3=1

save, and reboot


----------



## TheBlademaster01

Good to see it worked







. Did Gen 3 really net you some performance gains btw?


----------



## bigblock990

Quote:


> Originally Posted by *TheBlademaster01*
> 
> Good to see it worked
> 
> 
> 
> 
> 
> 
> 
> . Did Gen 3 really net you some performance gains btw?


Too soon to tell. Cant hurt though right


----------



## TheBlademaster01

True









I was curious since I used to do [email protected] on my machine in Linux as well.


----------



## geort45

Just need a quick yes or no... please

Do I need to do anything special to use a GTX 780 Ti on the z9? just swapped a super basic GT 210 or something for this bad boy but it doesnt go past the POST

Thanks









Sent from my iPhone using Tapatalk


----------



## TheBlademaster01

Yes, you need BIOS 3306 or later to POST with PCI-E 3.0 GPUs. It will hang at b2 otherwise.


----------



## geort45

Quote:


> Originally Posted by *TheBlademaster01*
> 
> Yes, you need BIOS 3306 or later to POST with PCI-E 3.0 GPUs. It will hang at b2 otherwise.


Thanks! I hope my ES CPU still works after that... I have a spare BIOS chip just in case


----------



## linol

Anyone here knows if it's possible to get a Samsung SM 951 M.2 pci-e (AHCI) on a m.2 to pci-e adaptercard to boot and work as OS-disk on our motherboard?

This is the adaptercard I will use
http://www.delock.com/produkte/S_89370/merkmale.html

The ssd I will use
http://www.amazon.com/Samsung-SM951-512GB-AHCI-MZHPV512HDGL-00000/dp/B00VELDBJ6


----------



## rvb3n

Hi everyone,

I've been an owner of this board for quite some time.

I've tried to update to Windows 10 with a clean installation but I've run into some problems.

Mainly the problems seems to be related with the sata controller.

In the device manger I get this message:



Until now I've installed only Windows 10 and the NVIDIA driver, no other drivers or utility from the ASUS web page as there's not a section for Windows 10.

Can you tell me how to solve this problem and which drivers / utilities to install (and where to get them)?


----------



## geort45

Quote:


> Originally Posted by *rvb3n*
> 
> Hi everyone,
> 
> I've been an owner of this board for quite some time.
> 
> I've tried to update to Windows 10 with a clean installation but I've run into some problems.
> 
> Mainly the problems seems to be related with the sata controller.
> 
> In the device manger I get this message:
> 
> 
> 
> Until now I've installed only Windows 10 and the NVIDIA driver, no other drivers or utility from the ASUS web page as there's not a section for Windows 10.
> 
> Can you tell me how to solve this problem and which drivers / utilities to install (and where to get them)?


Didnt you try win8 drivers?

Sent from my iPhone using Tapatalk


----------



## rvb3n

Yes I did, but when I try to install them I receive an error message and I can't go further!


----------



## lloyd mcclendon

I set this machine all up in 2012 / 2013, got it stable and just used it happily. I haven't been keeping up with this thread, there's too many different subjects going on.

Just last month I decided to clean the dust out and make a few upgrades. One sub-project was to flash the bios to the latest (v5802, just released on 10/15). In doing that, I totally forgot that it erases the settings I changed, and I of course also forgot what all the settings were.

So I've been working to get the machine back to a nice place and have been messing with the board a lot again lately. I still have many questions about the knobs in the bios, but my thoughts are still very disorganized, and really nobody out there seems to know much of anything anyway so asking is mostly pointless. ASUS support is 90% no help either, maybe I can try that channel again. Not for support per-se, but to get us some real documentation, real text about how to properly work this thing. With most things in life, product quality and product support quality vary inversely. The manual should be twice as thick, I've read it like 7 times.

My real immediate question that has been nagging since I purchased the board is: *WHAT does that tesla jumper really do?* Seriously I have never been able to find anything about that.

I feel like there is still much untapped potential in this board, but not being a seasoned motherboard engineer I'm not able to get it out, at least not without better documentation. Surely the guys that engineered this piece would have a few things setup differently than I do.

Do we have anyone here in the community that is a real expert on this thing?

I should add, I disabled all of the power managment crap just for high performance everywhere and that seems to have been a very good idea. Average wall consumption went up from about 350W to 425W. I haven't load tested anything yet as I am just getting over a few catastrophic hangs from I suspect forcing too much ddr speed.


----------



## geort45

Quote:


> Originally Posted by *TheBlademaster01*
> 
> Yes, you need BIOS 3306 or later to POST with PCI-E 3.0 GPUs. It will hang at b2 otherwise.


Im running 5304, just plugged the 780 and the error LEDS show 05..04..03...02..01...00...05 etc like a count down hahahha










Sent from my iPhone using Tapatalk


----------



## shredhead666

Hello everyone!

I just build myself a workstation with this motherboard and I was wondering if anyone has had this problem I'm having with this board. Basically It installs windows 7 with out any issues. But when I try to install 8, 8.1, 10 or server 2012 r2 it instantly goes into blue screen with "ACPI BIOS ERROR". I've tried booting with only 1 cpu, different memory kits, only 1 ram stick, different psus, swapping cpus, flashing to older bios all with the same result. I have also gone under the ACPI settings in the bios and have tried every configuration. I have also tried booting from usb stick as well as dvd drive. Right now I'm on bios 5802.

Here's my system:
cpus: E5 2680 x2
gpu: gtx 980
ram: gskill sniper 1600 64gb
storage: crucial 128gb ssd, Seagate 3tb hdd
psu: ocz zt 750 watt gold

Thanks very much!


----------



## bigblock990

Quote:


> Originally Posted by *shredhead666*
> 
> Hello everyone!
> 
> I just build myself a workstation with this motherboard and I was wondering if anyone has had this problem I'm having with this board. Basically It installs windows 7 with out any issues. But when I try to install 8, 8.1, 10 or server 2012 r2 it instantly goes into blue screen with "ACPI BIOS ERROR". I've tried booting with only 1 cpu, different memory kits, only 1 ram stick, different psus, swapping cpus, flashing to older bios all with the same result. I have also gone under the ACPI settings in the bios and have tried every configuration. I have also tried booting from usb stick as well as dvd drive. Right now I'm on bios 5802.
> 
> Here's my system:
> cpus: E5 2680 x2
> gpu: gtx 980
> ram: gskill sniper 1600 64gb
> storage: crucial 128gb ssd, Seagate 3tb hdd
> psu: ocz zt 750 watt gold
> 
> Thanks very much!


I am running linux on mine, so I can't help. I think @TheBlademaster01 might help, he seems super knowledgeable with these mobo's.


----------



## shredhead666

Thanks for your input I appreciate it









Actually I just solved this problem today so if anyone else who has this problem this might work for them too:

I bought a new bios chip from ebay and replaced the old one today and on the first try windows 10 installed without any issue








My board is a rev 1.04 and from what I read on earlier posts on this thread that was the first gen of boards out. I believe the actual bios chip was not fully able to use all the features of the newer .cap bios that are out. Just my theory and I hope it helps anyone else out there









Thanks Again!


----------



## rvb3n

Quote:


> Originally Posted by *shredhead666*
> 
> Thanks for your input I appreciate it
> 
> 
> 
> 
> 
> 
> 
> 
> 
> Actually I just solved this problem today so if anyone else who has this problem this might work for them too:
> 
> I bought a new bios chip from ebay and replaced the old one today and on the first try windows 10 installed without any issue
> 
> 
> 
> 
> 
> 
> 
> 
> My board is a rev 1.04 and from what I read on earlier posts on this thread that was the first gen of boards out. I believe the actual bios chip was not fully able to use all the features of the newer .cap bios that are out. Just my theory and I hope it helps anyone else out there
> 
> 
> 
> 
> 
> 
> 
> 
> 
> Thanks Again!


Which driver did you install on Windows 10? Do you have some device not recognized like I have?


----------



## shredhead666

I didn't install any drivers, everything just worked for me. And under device manager everything is detected. Have you tried installing the drivers from the asus website?


----------



## RobertFontaine

Good very early morning all,

I've been up most of the night reading and searching.... I have not read all 270 pages of this thread yet...
I've been through quite a few pages of this thread and a few others...

A Z9PE-D8 will be arriving on my doorstep in a few hours and hopefully will be accompanied (temporarily) with RAM and a cpu that will post the board.

I have some interesting requirements:

1) I'd like to run my cpus on this Board.
2) I have a pair of Xeon Phi processor cards (soon to be 3 or 4 with a little luck)
3) I will be running CentOS and maybe Xen (Undecided if I go Ring 0 hypervisor or do VMS inside of CentOS)

I have in my sweaty hands a pair of QA8Z B0 2665 8 Core 2.3 GHz chips, 8x4 kit of ECC 10600 slow but tested as good, an old reliable GTX580, My 2 Xeon PHI's are 31S1P's, the server version (no fan) and I have alphacool waterblocks on them but I don't have a pump and a rad yet. (next on the list right after successfully booting the beast), 1 stock lga 2011 heatsink and fan (lol) I could really use a second this morning but maybe if I just blow on the second one it will post before it overheats and crashes (not) .... A Core 9 Thermaltake enclosure, 1000 watt power supply, a bunch of drives, miscellaneous cable and plenty of coffee.

It looks like Xeon PHI and Tesla support were added in BIOS 3301. This is also the first of the .CAP BIOS releases.

Z9PE-D16 Formal BIOS 3301 release
1. Fixed system can not install OS on ASUS PIKE 2008/IMR.
2. Patch BIOS System Event Log record the error time.
3. Add "Above 4G Decoding" item in BIOS Advanced page -> PCI Subsystem Settings.
4. Update code for WHQL test & UEFI Firmware Certification test.

Back around page 7 of this 270+ page thread one of the members says that he had been running 2 - QA90 's on the 33xx release of the D8.
So I have hope.

The chinglish of 3401 below seems to say there was a conflict between the video and the 64bit (4g decoding) that they fixed. Seems like something I might want.

Z9PE-D16 Formal BIOS 3401 release
Prevent all VGA be 64bit resource will cause system hang or abnormal.

... not till the very newest release do I see something interesting that might be helpful (I have a GTX580 and will probably get a 7970 or 7990 at some point so new video card compatibility isn't a big deal if they are talking 980/Titan etc... ) ... So maybe I don't have an interest.

BIOS 5506 for Z9PE-D16 released to public
1. Improve compatibility with some 3D VGA Card and some GPU Cards.(NVIDIA K340,K520,K80,M40)
2. Improve compatibility for USB WHQL TEST and some other XHCI chip use.
3. Improve compatibility for PIKE card.
4. Improve compatibility with some add-on cards.
5. Improve AMI UEFI Variable Security.
6. Improve BMC stability.
7. Improve efficiency of power consumption.
8. Modify string description from "Lan1" to "LAN1".

So it kind of looks like 3401 might be my favourite BIOS for XEON PHI / ES B0 compatibilty. ???

The 3401for the D16 seems close to the equivalent of the 3506 for the D8.

Have people been successful running both cpus with QA8Z level chips at this BIOS revision level?

I have a bootable usb stick prepared with afudos, bupdater and the complete collection of D16 BIOS version.
I have downloaded the various versions of the manual in pdf.

I have a PCI error code card (not that 5A does me any good but at least I can differentiate between RAM and CPU causing POST problems.

Is there anything else I should have prepared to get to POST?

Should I just work my way backwards down the BIOS version until I can post with both cpus?

On a hopefully unrelated note...

Taobao has some nice prices on the 12 core xeons ES chips. Have there been decent success with getting 2 of them running on the Z9PE's?

Thanks in advance.. It's been months getting to this point and I'm hoping the magic smoke doesn't escape this afternoon.


----------



## RobertFontaine

... and what is the best search engine/translate for taobao?

I've been using google tranlate and taobao directly.
There is a wide variety of prices and some darn good ones in spots.

2.9Ghz 10 core 2670's are a nice step up from my 8 core 2665 2.3GHz and the 2687 3.4Ghz are almost desktop speeds.

I tried bhiner search and oddly the results were junk at twice the price. (interesting algorithm).
What kind of fee does the agent/reshipper take?

Is TaoBao barter or take it or leave it pricing?

Anyone simply using google translate and buying direct or does taobao require a chinese purchaser and a reship?

I haven't even posted for the first time and I'm already looking at upgrades.









thanks again


----------



## Jeffinslaw

Quote:


> Originally Posted by *RobertFontaine*
> 
> ... and what is the best search engine/translate for taobao?
> 
> I've been using google tranlate and taobao directly.
> There is a wide variety of prices and some darn good ones in spots.
> 
> 2.9Ghz 10 core 2670's are a nice step up from my 8 core 2665 2.3GHz and the 2687 3.4Ghz are almost desktop speeds.
> 
> I tried bhiner search and oddly the results were junk at twice the price. (interesting algorithm).
> What kind of fee does the agent/reshipper take?
> 
> Is TaoBao barter or take it or leave it pricing?
> 
> Anyone simply using google translate and buying direct or does taobao require a chinese purchaser and a reship?
> 
> I haven't even posted for the first time and I'm already looking at upgrades.
> 
> 
> 
> 
> 
> 
> 
> 
> 
> thanks again


When I purchased through Taobao, I used buychina.com as the middle man. On Taobao, you can only ship to a Chinese address. So buychina.com acts as the Chinese address so they can then ship the item to you.

-Jeffinslaw


----------



## shredhead666

Hey everyone!

Just wanted to let any one who was wondering know that the new Samsung 950 pro nvme m.2 ssd works on this board! I have it on a pcie x4 adapter card and I'm getting 2500 read and 1500 write! Just make sure you set the CSM in the bios for UEFI on pci-e


----------



## donkom

shredhead666, would this mean that any NVMe SSD should work? Been contemplating buying one for a while, but I've held off. Was this support added in firmware 5802?


----------



## shredhead666

Hey Donkom,

I'm pretty confident that they would work.

Here's intel's requirements for booting from nvme ssd, http://www.google.com/url?sa=t&rct=j&q=&esrc=s&source=web&cd=11&ved=0ahUKEwi5hd-IooTKAhVB6yYKHZYbC74QFghVMAo&url=http%3A%2F%2Fdownload.intel.com%2Fsupport%2Fssdc%2Fhpssd%2Fsb%2Fnvme_boot_guide_332098001us.pdf&usg=AFQjCNHBzJAqIZ-cLeqq303cPDID2NKoiw

Basically the bios needs to be uefi which in this board it is and it needs to have CSM support which this board also does. Asus didn't officially add support for nvme drives for this board but I don't think "official support" is needed. As long as the board has both uefi and CSM you should be good. From what I understand when a manufacturer add "support" basically the settings in the bios are already set to run off the nvme drive, so no tweaking required. Ive tested this nvme drive on a few other older boards besides this ws and they all booted and functioned properly. Just needed some tweaking in CSM lol.

I hope this helps









oh and I'm using bios 5802


----------



## ViperJPB

Anyone run into some odd issues with dual 980 TIs in SLI? Recently bought my second 980 TI and have been having some odd issues. In benchmarks and older games it scores as you would expect them to. They also score where they should individually. However in more modern games like Starwars Battlefront and Witcher III there are some odd issues with SLI. Witcher III literally gets half as many fps in SLI as it does in single card mode. 30s for SLI and near 60 for single with Ultra settings for both. Where in Battlefront it's a bit more subtle. I get a solid 60 always in solo play. But the larger multiplayer levels I get occasional drops by 7-10fps for quick moments. Feels like lag. However the same drops are not repeatable with SLI off and on the same settings. Solid 60fps with one card. So it's not lag. Have tried at least five different drivers. The last driver set introduced some seriously lag with SLI so I rolled back and that remedied the issue. This led me to believe it could be just software optimization. Latest drivers resolved that issue but I still get these occasional drops with SLI.

Any ideas? Am I having bandwidth issues on the board with one GPU on one CPU bus and the other on the second? I'm running the cards in slot 1 and slot 5 to give them some air. So using the first blue one and the third. Both are running at full 16x PCIe-3. Have never had SLI issues before with my previous three 680 GTs (2 in SLI and one for PhysX). Aside from moving the cards closer to both be on the same CPU is there anything I can try in the bios maybe? This machine is a workstation after all so SLI isn't a make or break but still a concern from the standpoint of possible hardware issues or config problems. Any help or suggestions would be greatly appreciated.


----------



## shredhead666

Hey viper,

Have you tried moving the second card to slot 3 or 4 just to see if you notice a difference? Also what bios are you on and what rev. is your board? I only as because my board is a rev. 1.04 and I had to replace the bios chip in order to fully use all the settings in the latest bios 5802. I have my system up right now running my two titan x's with no problems. They are running in slots 1 and 4. Give the slot change a shot and I hope this helps


----------



## ViperJPB

Thanks for the reply shredhead666. Yeah I think I am going to have to open her up again and swap slots to test this out. But if I'm not mistaken doesn't slot 4 force all slots to 8x? I'm not certain as to which bios I'm currently on or board version. I will check later and let you know. However I have had zero issues with it and any features. Also ran some major bios updates in preparation for the first 980 TI and the jump to Windows 10.


----------



## shredhead666

Your welcome Viper. Also if you put the card in slot 4 that will only force slot 3 into 8x. The easiest way for me to remember is if a black slot is used it cuts the blue slot above it to 8x lol. I hope it works out for ya


----------



## ViperJPB

Thanks again shred....interesting bit of info there. By the way my bios version is 5503. So I have not gone up all the way. I seem to recall on a higher bios it would restart twice before booting or something. So stuck with this one. My board is ver 1.0x. Thanks again for your thoughts. I may try it out this weekend.

Also a sidebar topic on the same lines....I have noticed this even with Win7 and 3 GTX 680 4gbs that whenever I switch to pcie-3 I my benchmarks dip slightly and is a difference of about 7 fps a second slower in 3.0 than 2.0. Odd.


----------



## donkom

Hey shredhead666, oh boy have I had a frustrating day. The Intel 750 NVMe will not show up as a boot device... but I'm certain there is some way to get this to work.

The Windows 10 installer sees the drive and can install the first set of files to it, but there is no way to get back to it. I tried installing Ubuntu on a secondary SSD to use it's bootloader to no avail. I tried Windows 10 on that same SATA SSD to just use it's bootloader but it didn't work. I might be doing something wrong in the process... ugh.

But you know what the strangest part is? The BIOS has PAGES of info on the 750 when it's installed. Never seen anything like it.






The BIOS clearly knows what to do with this device, yet I can't boot from it. The CSM is set for UEFI and Legacy, though I've set it to UEFI only with no luck....

What a frustrating day!


----------



## shredhead666

Hey donkom,

Ok let's start with some basic checks lol.
1. Which bios are you on?
2. In the CSM menu did you set pcie to UEFI only?
3. I'm not sure if this matters but which pci slot is the drive in?
4. When your gonna boot to the Windows installer, spam f8 to get into the boot menu and boot from your windows installer in uefi mode.

Don't worry bud we're gonna figure this out


----------



## donkom

Thanks shredhead666

1. 5802
2. In CSM, the PCIe is set to UEFI only.
3. The drive is in the bottom slot. Oddly enough, the system wouldn't POST if the drive was in the top slot.
4. Not sure if that would work... the Windows installer loads a lot of "stuff" that a bootloader doesn't Most bootloaders, so I've just learned, probe the BIOS and not anything more in-depth.... but the Windows installer loads up the necessary NVMe drivers.

Hmmm...


----------



## shredhead666

I would at least try to boot the uefi mode only because my Samsung 950 pro was acting weird in the windows installer if I didn't. Sometimes it would be detected and sometimes not. Also just out of curiosity, have you had either windows 8, 8.1 or 10 installed on this machine before this drive? I just wanna see if your board is having the same experience mine was where windows 8, 8.1 or 10 where not installing correctly and 7 would work flawlessly.

Lastly looking at your bios pic there, try with the storage option on legacy. Mine works with it on legacy and for some reason when in UEFI all my other drives are not detected.

Hope this helps


----------



## donkom

I wish that did it shredhead666, but no luck.

When I change the storage OpRom to legacy, I lose the BIOS entires for the 750 data. Nothing works.

I've also attached an error that I see when trying to use the windows bootloader to pick up this drive... it's complaining it can't find a file, but in reality it can't find ANY file - the NVMe drive is invisible until such drivers are loaded or the BIOS smartens up.

Ugh!


----------



## shredhead666

Don't worry I still have a few more ideas









I would say first as a test install Windows 10 on a regular sata drive just to make sure it will install on the board. If it doesn't install correctly then I'm pretty sure what the problem is. Also check the Intel website to see if there is a firmware update for the drive. And lastly what revision is your board?


----------



## shredhead666

I just remembed something, use a program called rufus to make a uefi installer for Windows on a usb drive. That might work too lol


----------



## donkom

Some great ideas there, shredhead666!

Windows 10 installs just fine on a regular SATA drive. No issues.

There was a firmware update for the drive - good call! Though it fixed nothing.

The motherboard is revision 1.04

I tried the rufus utility - neat, but it'll only boot into FreeDOS or an installation ISO. It doesn't have the capacity to act like a bootloader beyond that. From FreeDOS, the NVMe drive is not visible.

Grarg!


----------



## shredhead666

Alright so we got some good news!









I only suggested Rufus because with my I guess standard usb Windows installer Windows would fail on install on the nvme drive but when I used Rufus to make a uefi installer it worked lol.

Ok so here's another thought, what CPUs are you using? Do you know what stepping they are?


----------



## donkom

Curious why you want to know about the CPUs!

2x E5-2697v2 processors. Stepping 4, according to CPU-Z


----------



## shredhead666

I'm sorry I meant revision not stepping lol.

From what I read posted much earlier in this thread is that CPUs with b0 revision have a had time with newer hardware.


----------



## donkom

Sadly, these are among the last CPU models designed for this board. Revision C0/C1... and the frustration continues.


----------



## shredhead666

No that's a good thing, steppingstone c0 and c1 are great and that means the CPUs aren't the problem









Ok so I just did a reinstall on my system with an extra nvme drive I had and I got the same result where with my normal usb installer it failed but the Rufus uefi usb it worked. The only other option I can think is that I replaced my bios chip with a brand new one.

From what I've read on this forum and a few others the bios chip on these older rev boards doesn't support or recognize all the bios features that the newer bios releases offer. The newer ones are the 4 digit ones such as 5802 where the older ones are 3 such as 302 for example.


----------



## shredhead666

Stepping not steppingstone in my last post lol. Stupid auto correct XD


----------



## shredhead666

This is where I bought mine in case your interested:

http://www.ebay.com/itm/261069164961?_trksid=p2060353.m2749.l2649&ssPageName=STRK%3AMEBIDX%3AIT


----------



## donkom

The problem is, the Windows 10 installer doesn't have an issue. It detects the drive perfectly fine, installs all the needed files and then reboots the system. This is when the BIOS doesn't allow the Intel 750 to be a boot device, and it can't load the OS to finish the installation after the first reboot. Rufus, as I understand it, wouldn't help... as it would only allow me to relaunch the ISO containing the installer, and not function as a bootloader to work around the problem...

I had to buy a BIOS directly from ASUS when I got my system, as the default BIOS couldn't detect my newer CPUs. So, the BIOS chip in my system is newer than the one it shipped with.

Welcome to my nightmare.


----------



## lloyd mcclendon

Anyone running linux have working S3 resume? I had it working before, but now with the latest bios AND latest kernel, I can't get it to resume. Stuck on code E0, microcode not found. Not sure if there's a simple fix on my end for that.

I'm about back to stable finally, suspend being the last item on the list. I had a just barely bad DIMM and swapping the AMD card for this sweet GTX 970 has really cleared up a lot of things.

I am just stuck on microcode not found. I have a bunch of things to rule out software wise but i'm concerned it's the bios. If anyone has S3 resume working, what settings do you have in bios, acpi auto configure or what else??


----------



## Donkey1514

Quote:


> Originally Posted by *donkom*
> 
> The problem is, the Windows 10 installer doesn't have an issue. It detects the drive perfectly fine, installs all the needed files and then reboots the system. This is when the BIOS doesn't allow the Intel 750 to be a boot device, and it can't load the OS to finish the installation after the first reboot. Rufus, as I understand it, wouldn't help... as it would only allow me to relaunch the ISO containing the installer, and not function as a bootloader to work around the problem...
> 
> I had to buy a BIOS directly from ASUS when I got my system, as the default BIOS couldn't detect my newer CPUs. So, the BIOS chip in my system is newer than the one it shipped with.
> 
> Welcome to my nightmare.


may I suggest modding the bios by adding the proper NVMe modules - http://www.overclock.net/t/1571271/tutorial-how-to-add-nvme-support-on-any-ami-uefi-bios-with-an-intel-chipset


----------



## koooowweeee

Hello.

Well I've just been given one of these boards and 2 xeons. Very happy boy hehe

However I have 2 problems if anyone can help me out please.

First off it seems slot 6 and 7 don't work? I've swapped the ram about (all sticks are the same) but still doesn't work. Could it be faulty? Seems odd that it's 6 and 7 and not just 1 or 2 together.

Second question is. I've got 2 corsair h80 fitted to them. 1 is nice and quiet and the second one's fans a at full speed all the time. Is there a way to control them with the board or any other programs?

Last question now. Can I disable the second CPU? I only game so there's no point in having the second CPU? Will save power and heat. Plus I can turn off the fans on the second h80 

Any help on them 3 questions would be a great help.

Thanks

Just realised this thread isn't for my board. I have the WS not the D8


----------



## celemine1gig

@lloyd:
Same thing here with S3 under Win7. Using the latest BIOS. Hibernate isn't working, correctly, too.

Will have to test a bit, but could be a simple BIOS bug. We'll have to see. Someone already reported this to ASUS?
I have only limited time for this at the moment.

We should check if the ACPI autoconfiguration option in BIOS somehow makes a difference. If this is a bug, it could be.


----------



## SRPlus

Hi,

Been a while since I visited this thread, so Hi to all!

Two questions, hopefully someone can confirm both.

*One.*

I am thinking about buying a gsync monitor. I have 4x GTX Tians ( older 2688 core)

The system is mainly used for 3ds max and production rendering with Iray, but occasional gaming.

I have read on this thread that there have been problems with 4-way SLI and surround set up's where only 2-way or 3 way SLI is selectable.

I don't use surround, but would the same problem happen with a g-sync monitor in 4-way sli?

4-way SLI works fine on my system, never had any problems, but never tried g-sync before.

Maybe what I am asking is completely illogical, but thought I would ask before purchasing the monitor.

Anyone using 4-way SLI and G-sync with this MoBo? Any problems?

*Two.*

I cant remember the exact date, but there was a windows update some time in December that included driver updates for the C602 chipset.

Before the update all 4 GPU's ran @ pcie Gen 3.0, after the update the primary card only runs at pcie 2.0 and the other three at pcie 3.0.

Tried lots of things, cant correct this. I am using The Nvidia control panel System info to check this, but GPU-Z gives the same info.

Obviously I already know how to apply the force enable Gen 3.0 Patch on the X79 platform in Windows 10 Pro x64, or it wouldn't have been working before the updates.

I've already checked the registry and the correct RMPcieLinkSpeed setting is there on all cards. Although I have always used the method of copying the force enable gen 3 exe to the main C drive and running it from the command line as administrator which is much simpler.

All pcie lanes are set to Gen 3.0 in the bios.

Anyone else found that their primary card in a 4 GPU set up is only running now at pcie 2.0 on the Z9PE-D8 WS?

Cheers!


----------



## Venoms

• 2x Intel Xeon E5-2687W v2

• Asus Z9PE-D8 WS

• 3x SLI Titan X

• ASUS 27" PG278Q 3D

• 64GB DDR3 1866 CORSAIR Dominator Platinum (8x 8GB sticks)

Running Windows 10...

Yes as I am reading here it looks like I'm screwed. I'm running 3 ROG Swift Gsync monitors and I cant seem to get nvidia surround to work. Video cards do not seem to be able to perform as expected.

As soon as I go to nvidia surround it gives me a "connection error" even though (according to nvidia support) everything is connected correctly.

Is there any hope for this, anyone else having this issue? Or is there no choice but to roll back to roll back to Windows 8 (which I've heard nothing but negative things about, but I understand it has better hyper threading performance)???


----------



## celemine1gig

Concerning the S3 and S4 issues:
This seems to be linked to some BIOS settings. I had just reflashed the latest BIOS today and changed settings, step by step.

At first, I did not get the "0E" hang error, when waking from S3 and/or S4. After some more settings, the error was back. So this is somehow linked to some specific BIOS setting(s).
Can't tell which one exactly, however, just yet.


----------



## Roaches

Hi guys, I'm wondering if this board is worth buying into? Xeon E5-2670s SR0KX chips are currently selling for peanuts right now and I'm itching to grab a couple for to give an oomph to my current CAD workstation build.


----------



## Toqi

hi guys
2x E5-2670 and ASUS Z9PA-D8/iKVM this mobo worked 2x GTX 970 sli ?


----------



## Migonacci

I have a curious issue that started happening. I got a roccat tyon mouse and every time I boot the comp it hangs at the device list of the boot sequence.....and it curiously lists 2 keyboards instead of one ( the mouses' extra set of keys shows up as a 2nd keyboard). When I unplug the mouse the computer continues to boot into windows 10. I'm running a 5601 bios revision.

Also for what it's worth this started happening after I upgraded from windows 7 to win 10....even though I don't think the OS should have any bearing on the boot up sequence from the mobo.

Anyone have any ideas?

thanks


----------



## SRPlus

I also have this MoBo and Roccat tyon, running on Windows 10 Pro x64, no problems.

Bios update 5701 and 5802 further improve compatibility problems with USB keyboard and mice, so you should try the latest 5802 bios first.

Did you make any changes to where you plug your USB devices in?

Was the mouse your only upgrade? No USB 3.0 keyboard?

Also, if the bios update doesn't do the trick, put your old mouse back in so you can get into the O/S.

Then change your polling rate in the Tyon mouse settings down to 500hz.

I have mine plugged in on the USB headers at the top near the PS/2 sockets. Never had any problems.

Make sure your USB driver is installed correctly.

Last thing, and this one takes some time, make sure your chipset is up to date. Microsoft issued chipset updates for this mobo back in December 2015, and for some reason most of them do not install automatically, so you have to go down the list in system devices, right clicking each component, selecting update driver software, and search automatically. Your screen will go blank a few times when updating the PCIE related devices, and some may require a restart so best to note your place down the list when it asks for a reboot.

I think the Bios update will probably solve the problem though...


----------



## 4hwgenxx

Hi guys there is a list of best setting for this board to get maximum performace possible?


----------



## PLan2013

Might be a dumb question but are there various versions of this motherboard? Taobao keeps showing two slightly different boards for Z9PE-D8 WS -


Here is a Taobao listing again with two slightly different board variations -

http://world.taobao.com/item/38902995321.htm?spm=a312a.7700714.0.0.vbQmxt#detail

EDIT:

So I guess I'm looking at models Z9PE-D8 WS and Z9PE-D8?


----------



## Venoms

I had a buddy put my machine together for me, but I've been having some issues. Here's my specs:

• ASUS Z9PE-D8 WS Dual LGA 2011 Motherboard
• 64GB CORSAIR Dominator Platinum DDR3 1866 RAM
• 3x SLI Evga Titan X Hydro
• 3x ASUS ROG Swift PG278Q (wanting to run nVidia surround 2650x1440ea)

For one thing, I'm only getting 1600MHz out of the RAM... From what I hear you need to overclock the board in order to get the 1866, but I've also heard that caused issues.

My BIGGER problem is I can't get nVidia surround to work. I've spoken to nVidia, and they are clueless. They've had me roll back drivers, try different Display Ports, etc. I rolled back from Windows 10 back to Windows 7 since Asus does not have drivers for Windows 10 yet, but the exact same issue is occurring.

Every time I try to activate nVidia surround I get an error: "Display Connection Error"


I am now suspecting that the actual configuration of the cards themselves may have something to do with this. My friend started the SLI from PCI 2 down instead of plugging into PCI slot 1. This will prevent me from getting to x16, but I'm not sure if this is going to be related to the surround or not? Anyone else have any ideas? Should I swich my SLI from PCI 2, 4, 6 to PCI 1, 3, 5?

So far I have a significant investment in this rig, but I suspect I am not seeing the performance that it should be capable of as of yet.

Any help would be much appreciated.

Thanks!


----------



## SRPlus

Quote:


> Originally Posted by *Venoms*
> 
> I had a buddy put my machine together for me, but I've been having some issues. Here's my specs:
> 
> • ASUS Z9PE-D8 WS Dual LGA 2011 Motherboard
> • 64GB CORSAIR Dominator Platinum DDR3 1866 RAM
> • 3x SLI Evga Titan X Hydro
> • 3x ASUS ROG Swift PG278Q (wanting to run nVidia surround 2650x1440ea)
> 
> For one thing, I'm only getting 1600MHz out of the RAM... From what I hear you need to overclock the board in order to get the 1866, but I've also heard that caused issues.
> 
> My BIGGER problem is I can't get nVidia surround to work. I've spoken to nVidia, and they are clueless. They've had me roll back drivers, try different Display Ports, etc. I rolled back from Windows 10 back to Windows 7 since Asus does not have drivers for Windows 10 yet, but the exact same issue is occurring.
> 
> Every time I try to activate nVidia surround I get an error: "Display Connection Error"
> 
> 
> I am now suspecting that the actual configuration of the cards themselves may have something to do with this. My friend started the SLI from PCI 2 down instead of plugging into PCI slot 1. This will prevent me from getting to x16, but I'm not sure if this is going to be related to the surround or not? Anyone else have any ideas? Should I swich my SLI from PCI 2, 4, 6 to PCI 1, 3, 5?
> 
> So far I have a significant investment in this rig, but I suspect I am not seeing the performance that it should be capable of as of yet.
> 
> Any help would be much appreciated.
> 
> Thanks!


As for the surround problem, I don't use it so cant help much there.

But from your photo, I see you have the cards plugged in pci-e slots 2, 4, and 6, these are only x8 slots, you need to plug the cards into the blue x16 slots for better performance.

Also are you using the force enable Gen 3 patch for this motherboard?

As for memory, if you are using the V1 E5-2600 CPU's you may not be able to go above 1600, V2's upwards I think are ok. This MoBo is very temperamental about the memory you used. Xeons usually prefer ECC R memory. But I know some non EC R memory has also been successful..


----------



## Venoms

Quote:


> Originally Posted by *SRPlus*
> 
> As for the surround problem, I don't use it so cant help much there.


I plan to use for gaming and 3D Maya/Mudbox work. IF I can get the surround working, would there be a specific set up that I should toggle between for best 3D workstation performance vs gaming performance?
Quote:


> Originally Posted by *SRPlus*
> 
> But from your photo, I see you have the cards plugged in pci-e slots 2, 4, and 6, these are only x8 slots, you need to plug the cards into the blue x16 slots for better performance.


Yeah, sounds like we'll need to fix his mistake. For some reason he thought that if you use three cards, these were the slots he needed to use. But it appearst that 1,3 and 5 are the expected PCI slots for 3x SLI. Which who knows, maybe that is related to the surround issue I'm dealing with.
Quote:


> Originally Posted by *SRPlus*
> 
> Also are you using the force enable Gen 3 patch for this motherboard?


What is that? Where can I get it? I really haven't done anything other than update the bios and drivers that were on the Asus support site, so please catch me up on anything that I should be doing to maximize performance!








Quote:


> Originally Posted by *SRPlus*
> 
> As for memory, if you are using the V1 E5-2600 CPU's you may not be able to go above 1600, V2's upwards I think are ok. This MoBo is very temperamental about the memory you used. Xeons usually prefer ECC R memory. But I know some non EC R memory has also been successful..


Ah yeah, that's kind of what I gathered. Does not sound like it's worth the hassle for the extra 233MHz.


----------



## 4hwgenxx

Hi guys someone figured out on "how to enable" pciex 3.0 on this motherboard? this is my specs:

z9ped8ws

2x xeon 2670 (revision C2)

64gb 8x8gb of non-ecc kingston 1866 memory (running at their top speed 1866)

evga gtx 980 ti

i have enable GEN3 on the bios, but nothing, still stuck to pciex 2.0 16x

i have tried "force-enable-gen3" patch but it seems to not work, it run but close immidiatly...i dunno what to do... :/


----------



## Migonacci

Quote:


> Originally Posted by *SRPlus*
> 
> I also have this MoBo and Roccat tyon, running on Windows 10 Pro x64, no problems.
> 
> Bios update 5701 and 5802 further improve compatibility problems with USB keyboard and mice, so you should try the latest 5802 bios first.
> 
> Did you make any changes to where you plug your USB devices in?
> 
> Was the mouse your only upgrade? No USB 3.0 keyboard?
> 
> Also, if the bios update doesn't do the trick, put your old mouse back in so you can get into the O/S.
> 
> Then change your polling rate in the Tyon mouse settings down to 500hz.
> 
> I have mine plugged in on the USB headers at the top near the PS/2 sockets. Never had any problems.
> 
> Make sure your USB driver is installed correctly.
> 
> Last thing, and this one takes some time, make sure your chipset is up to date. Microsoft issued chipset updates for this mobo back in December 2015, and for some reason most of them do not install automatically, so you have to go down the list in system devices, right clicking each component, selecting update driver software, and search automatically. Your screen will go blank a few times when updating the PCIE related devices, and some may require a restart so best to note your place down the list when it asks for a reboot.
> 
> I think the Bios update will probably solve the problem though...


Thanks for the insight SRplus; I will try the things you mentioned to see if that fixes the issue.


----------



## SRPlus

Quote:


> Originally Posted by *4hwgenxx*
> 
> Hi guys someone figured out on "how to enable" pciex 3.0 on this motherboard? this is my specs:
> 
> z9ped8ws
> 
> 2x xeon 2670 (revision C2)
> 
> 64gb 8x8gb of non-ecc kingston 1866 memory (running at their top speed 1866)
> 
> evga gtx 980 ti
> 
> i have enable GEN3 on the bios, but nothing, still stuck to pciex 2.0 16x
> 
> i have tried "force-enable-gen3" patch but it seems to not work, it run but close immidiatly...i dunno what to do... :/


1 : Go to the link below and download the NVidia force enable Gen 3 patch.

http://nvidia.custhelp.com/app/answers/detail/a_id/3135/~/geforce-gen3-support-on-x79-platform

If you search on forums some will tell you to edit the registry but its completely unnecessary, and this way is quicker.

2: Copy the force-enable-gen3.exe to your main C: directory.

3: Open the command prompt as an administrator.

Type : cd C:/
Then type : force-enable-gen3.exe

Your installed GPU(s) should be listed.

Type exit.

Restart you PC.

Next time you check in the Nvidia control panel system info, or GPU-Z , it should say pcie 3.0

In GPU-Z you will only get pcie 3.0 x16 at 3.0 if the GPU's are under load.

Make sure in the bios your GPU's are set to Gen 3 not auto!


----------



## SRPlus

Quote:


> Originally Posted by *Venoms*
> 
> I plan to use for gaming and 3D Maya/Mudbox work. IF I can get the surround working, would there be a specific set up that I should toggle between for best 3D workstation performance vs gaming performance?
> Yeah, sounds like we'll need to fix his mistake. For some reason he thought that if you use three cards, these were the slots he needed to use. But it appearst that 1,3 and 5 are the expected PCI slots for 3x SLI. Which who knows, maybe that is related to the surround issue I'm dealing with.
> What is that? Where can I get it? I really haven't done anything other than update the bios and drivers that were on the Asus support site, so please catch me up on anything that I should be doing to maximize performance!
> 
> 
> 
> 
> 
> 
> 
> 
> Ah yeah, that's kind of what I gathered. Does not sound like it's worth the hassle for the extra 233MHz.


I will have to answer these questions tomorrow evening, been working at the pc all day  Unless someone else can help out sooner .


----------



## Venoms

Quote:


> Originally Posted by *SRPlus*
> 
> I will have to answer these questions tomorrow evening, been working at the pc all day  Unless someone else can help out sooner .


Any more thoughts on this?

Just to confirm, for triple Titan X SLI with dual CPUs I'll want to go PCI-E 1,3,5 on this board correct?

Also, for using Maya, VRay, Mudbox etc. are there any specific settings or drivers that I should be using to maximize performance? And what about getting the best I can for gaming?


----------



## SRPlus

Quote:


> Originally Posted by *Venoms*
> 
> I plan to use for gaming and 3D Maya/Mudbox work. IF I can get the surround working, would there be a specific set up that I should toggle between for best 3D workstation performance vs gaming performance?
> Yeah, sounds like we'll need to fix his mistake. For some reason he thought that if you use three cards, these were the slots he needed to use. But it appearst that 1,3 and 5 are the expected PCI slots for 3x SLI. Which who knows, maybe that is related to the surround issue I'm dealing with.
> What is that? Where can I get it? I really haven't done anything other than update the bios and drivers that were on the Asus support site, so please catch me up on anything that I should be doing to maximize performance!
> 
> 
> 
> 
> 
> 
> 
> 
> Ah yeah, that's kind of what I gathered. Does not sound like it's worth the hassle for the extra 233MHz.


It depends on what method you use for rendering the final image in Maya. If you are using GPU based rendering like Iray with multiple cards, its best to have only one GPU used for the monitors, and the other cards solely for GPU rendering. This way, you can select GPU 2 and 3, and leave your first GPU free to manage the operating system and applications, so you can still work without a performance hit to you displays, and sometimes freezing.

The above method is no good of course for surround gaming, as you would need to keep swapping over your display inputs whenever you want to game in surround.

If you are using CPU based rendering like Mental Ray, it doesn't matter if you set up the GPU display outputs for surround gaming.

SLI must definitely be turned off for GPU rendering with Max / May, and PhysX set to CPU in the NVidia control panel. It can still work in SLI but can cause instability, and if you are working on large scenes that use a lot of GPU memory, it can lock up your workstation completely.

Need to know what renderer you use first.

As for the pcie slots, whoever built the machine for you, only has to look in the user manual that came with the motherboard or download the online one. All the blue slots are x16, 1, 3, 5, 7. Slots 1 & 3 are controlled by CPU1 and 5 and 7 by CPU2.

You should also make sure your pcie slots are set to Gen3 in the bios, not to Auto. And use the Force enable Gen3 patch I gave instructions to a few posts back. This method is only needed in Windows 10. Previous O/S versions will run the executable correctly from any location.

I'm not 100% sure, but if you are using E5-2600 V2's it should already run at pcie 3.0 without the patch. You can check this by going into your Nvidia control panel, and clicking on the system information link at the bottom left. Then scrolling down the information field to the bottom.


----------



## 4hwgenxx

Quote:


> Originally Posted by *SRPlus*
> 
> 1 : Go to the link below and download the NVidia force enable Gen 3 patch.
> 
> http://nvidia.custhelp.com/app/answers/detail/a_id/3135/~/geforce-gen3-support-on-x79-platform
> 
> If you search on forums some will tell you to edit the registry but its completely unnecessary, and this way is quicker.
> 
> 2: Copy the force-enable-gen3.exe to your main C: directory.
> 
> 3: Open the command prompt as an administrator.
> 
> Type : cd C:/
> Then type : force-enable-gen3.exe
> 
> Your installed GPU(s) should be listed.
> 
> Type exit.
> 
> Restart you PC.
> 
> Next time you check in the Nvidia control panel system info, or GPU-Z , it should say pcie 3.0
> 
> In GPU-Z you will only get pcie 3.0 x16 at 3.0 if the GPU's are under load.
> 
> Make sure in the bios your GPU's are set to Gen 3 not auto!


I will give a try

I have a question guys. I'm looking to buy a couple of "e5 2667 v2 es". My problem is...there are problem witch the ES? I dunno maybe may crash during a render or other stuff? And another thing, the 2667v2 es like the "non-es" will boost to 3.6ghz for all 8 core?

Edit: the metod for unlock pciex 3.0 worked but only for gpu2, gpu1 is still at 2.0


----------



## SRPlus

Quote:


> Originally Posted by *4hwgenxx*
> 
> I will give a try
> 
> I have a question guys. I'm looking to buy a couple of "e5 2667 v2 es". My problem is...there are problem witch the ES? I dunno maybe may crash during a render or other stuff? And another thing, the 2667v2 es like the "non-es" will boost to 3.6ghz for all 8 core?
> 
> Edit: the metod for unlock pciex 3.0 worked but only for gpu2, gpu1 is still at 2.0


I have the same problem since the windows 10 update in December that included updates for Xeon's and the c602 chipset.

Primary GPU always runs at pci-e 2.0, the other three GPU's run at pci-e 3.0.

Before Microsoft's wonderful update, all GPU's ran at 3.0 after the force-enable-gen3 patch.

Don't expect a fix soon, I spoke to a lady at Microsoft yesterday about it, and her opinion was that Microsoft don't release updates with glitches!

I find this comical, seeing as the whole purpose of updates is to correct glitches not found in previous updates, security improvements, so what she said is a complete contradiction to the purpose of updates )

Its definitely a Microsoft problem. I know that all cards were running at pci-e 3.0 before the Microsoft updates as I checked, after, only 3 cards run at 3.0. I have checked in the registry, and the correct fields are there for each GPU, so this is something Microsoft have to fix.

I suspect that they only tested their c602 / Xeon updates on one of intel's C602 motherboards, which only have 3 x16 slots.. but that's just guessing.

Until more people complain about the problem, which some might not even be aware of, Microsoft will probably continue to ignore it.... and their online chat support reps are obviously useless! She even said that I cant use Xeons with Windows 10 Pro x64 as is doesn't support multiple CPU's and that I need to use the Enterprise edition! Windows 10 Pro x64 can use up to two physical CPU's and up to 256 cores, shows the amount of training these guys and girls get..


----------



## 4hwgenxx

I'm gonna installa Windows 10 Enterprise right now! lol ahahahaha...by the way, so it's a problem of windows 10? so if roll back my glorius win 7 the problem do not show up? i use my sistem for work and gaming, and the only thing for i put in windows 10 is because remedy said that Quantum Break will run only in win 10...and this make my heart stop...that windows 7 still rocks and god bless microsoft because they do not know what their doing...


----------



## SRPlus

Quote:


> Originally Posted by *4hwgenxx*
> 
> I'm gonna installa Windows 10 Enterprise right now! lol ahahahaha...by the way, so it's a problem of windows 10? so if roll back my glorius win 7 the problem do not show up? i use my sistem for work and gaming, and the only thing for i put in windows 10 is because remedy said that Quantum Break will run only in win 10...and this make my heart stop...that windows 7 still rocks and god bless microsoft because they do not know what their doing...


The same, mainly work and occasional gaming.

I don't know, they may have applied the same updates to Win7 by now...

This is also Asus's fault. Win 10 has been around long enough now for them to release new chipset drivers for this mobo.

As for gaming, in some games on Win7 Pro x64, like bioshock infinite, metro last light and skyrim, I was getting microstutter using 4-way sli, but since upgrading to Win10Pro, its completely gone and all games run sickeningly smooth, so I am sticking to win 10 even if one card runs a fraction slower because of the pcie speed.

I hope Microsoft correct the problem, but from the knowledge and attitude of their support rep yesterday, I'm thinking it may be a very long time. I told her can she report the problem to the development team, she said yes, butt I got the feeling it was all going in one eyeball and out the other :-///


----------



## HiTekJeff

Quote:


> Originally Posted by *RushiMP*
> 
> Thanks alot, let me see if I can duplicate some of your success.


Quote:


> Originally Posted by *Venoms*
> 
> Any more thoughts on this?
> 
> Just to confirm, for triple Titan X SLI with dual CPUs I'll want to go PCI-E 1,3,5 on this board correct?
> 
> Also, for using Maya, VRay, Mudbox etc. are there any specific settings or drivers that I should be using to maximize performance? And what about getting the best I can for gaming?


You should do a search for Workstation builds on this site, as you will find some, including mine with many details and information you seek. My link is below, be sure to read the FAQ and Electrical sections.

http://www.overclock.net/t/1368079/build-log-3d-render-bear


----------



## NapalmV5

Quote:


> Originally Posted by *4hwgenxx*
> 
> I will give a try
> 
> I have a question guys. I'm looking to buy a couple of "e5 2667 v2 es". My problem is...there are problem witch the ES? I dunno maybe may crash during a render or other stuff? And another thing, the 2667v2 es like the "non-es" will boost to 3.6ghz for all 8 core?
> 
> Edit: the metod for unlock pciex 3.0 worked but only for gpu2, gpu1 is still at 2.0


Quote:


> Originally Posted by *SRPlus*
> 
> I have the same problem since the windows 10 update in December that included updates for Xeon's and the c602 chipset.
> 
> Primary GPU always runs at pci-e 2.0, the other three GPU's run at pci-e 3.0.
> 
> Before Microsoft's wonderful update, all GPU's ran at 3.0 after the force-enable-gen3 patch.
> 
> Don't expect a fix soon, I spoke to a lady at Microsoft yesterday about it, and her opinion was that Microsoft don't release updates with glitches!
> 
> I find this comical, seeing as the whole purpose of updates is to correct glitches not found in previous updates, security improvements, so what she said is a complete contradiction to the purpose of updates )
> 
> Its definitely a Microsoft problem. I know that all cards were running at pci-e 3.0 before the Microsoft updates as I checked, after, only 3 cards run at 3.0. I have checked in the registry, and the correct fields are there for each GPU, so this is something Microsoft have to fix.
> 
> I suspect that they only tested their c602 / Xeon updates on one of intel's C602 motherboards, which only have 3 x16 slots.. but that's just guessing.
> 
> Until more people complain about the problem, which some might not even be aware of, Microsoft will probably continue to ignore it.... and their online chat support reps are obviously useless! She even said that I cant use Xeons with Windows 10 Pro x64 as is doesn't support multiple CPU's and that I need to use the Enterprise edition! Windows 10 Pro x64 can use up to two physical CPU's and up to 256 cores, shows the amount of training these guys and girls get..


guys you must be on bios 5802 hence gpu @ slot 1 is at pcie2.0

ioh 0: iou2 pcie port and port 2a link speed are missing from bios 5802

contact asus


----------



## SRPlus

Quote:


> Originally Posted by *NapalmV5*
> 
> guys you must be on bios 5802 hence gpu @ slot 1 is at pcie2.0
> 
> ioh 0: iou2 pcie port and port 2a link speed are missing from bios 5802
> 
> contact asus


I've been using 5802 since the day it was released, October last..

The pcie problem only started happening since the Decembers updates.

I always check the link speed after an Nvidia driver update and using the patch, so 100% sure this started after the mid December updates that included Xeon and c602 chipset udates.

If you are using an older Bios and yours is working ok with the latest Win 10 updates, maybe its a conflict somewhere....


----------



## SRPlus

Quote:


> Originally Posted by *NapalmV5*
> 
> guys you must be on bios 5802 hence gpu @ slot 1 is at pcie2.0
> 
> ioh 0: iou2 pcie port and port 2a link speed are missing from bios 5802
> 
> contact asus


I will give Asus Tech Support a try, after all , its about time they released some new chipset drivers for this MoBo!

But I hate contacting Asus technical support.

The few times I have needed their help, they have been completely useless, and I end up solving the problem myself without following any of their advice..

They usually either say reset the bios / cmos, or reinstall your operating system, never had to do any of those to correct the problems...

or

They will copy past some pre written answer that has no relevance whatsoever to what I just asked. Nightmare!


----------



## NapalmV5

thats why i said u guys contact asus i dont have the nerves nor patience to









i was on win10 last year when they released 5802 - gpu1 @ pcie2.0

i just put 5802 on win7 still gpu1 @ pcie2.0

the iou2 pcie port is still there just port 2a link speed is missing


----------



## SRPlus

Quote:


> Originally Posted by *NapalmV5*
> 
> thats why i said u guys contact asus i dont have the nerves nor patience to
> 
> 
> 
> 
> 
> 
> 
> 
> 
> i was on win10 last year when they released 5802 - gpu1 @ pcie2.0
> 
> i just put 5802 on win7 still gpu1 @ pcie2.0
> 
> the iou2 pcie port is still there just port 2a link speed is missing


Wellp, I've submitted a support request )

Already set for a disappointing and completely useless answer :-///


----------



## 4hwgenxx

Let me know what they answer


----------



## SRPlus

Quote:


> Originally Posted by *NapalmV5*
> 
> thats why i said u guys contact asus i dont have the nerves nor patience to
> 
> 
> 
> 
> 
> 
> 
> 
> 
> i was on win10 last year when they released 5802 - gpu1 @ pcie2.0
> 
> i just put 5802 on win7 still gpu1 @ pcie2.0
> 
> the iou2 pcie port is still there just port 2a link speed is missing


I got my answer back.

Thankfully got one of the Europe reps.

He seems to think its a compatibility problem with Win 10 and the bios, that people using win 10, and the latest chipset / Xeon updates from Microsoft, are the only ones not seeing the pcie link speed selection for pcie slot one in the bios, and that win 10 has changes the bios options.

Anyone using Win 7 or 8 and not seeing the link speed options gen2, gen3 auto, for the first pcie slot, using bios 5802?

He basically said we have to wait for Win 10 drivers and bios to be released for this MoBo, and that its not compatible with Win 10,but that he cannot see any current plans for this in development at Asus...

It was compatible with win 10 before the Microsoft December updates!


----------



## 4hwgenxx

Quote:


> Originally Posted by *SRPlus*
> 
> I got my answer back.
> 
> Thankfully got one of the Europe reps.
> 
> He seems to think its a compatibility problem with Win 10 and the bios, that people using win 10, and the latest chipset / Xeon updates from Microsoft, are the only ones not seeing the pcie link speed selection for pcie slot one in the bios, and that win 10 has changes the bios options.
> 
> Anyone using Win 7 or 8 and not seeing the link speed options gen2, gen3 auto, for the first pcie slot, using bios 5802?
> 
> He basically said we have to wait for Win 10 drivers and bios to be released for this MoBo, and that its not compatible with Win 10,but that he cannot see any current plans for this in development at Asus...
> 
> It was compatible with win 10 before the Microsoft December updates!


fantastic...so There is a problem but probably they dont fix it...wonderfull... :/


----------



## SRPlus

Quote:


> Originally Posted by *4hwgenxx*
> 
> fantastic...so There is a problem but probably they dont fix it...wonderfull... :/


If someone can confirm this problem also exists in Win 7 & 8, its just Asus passing the buck, and need to sort their bios out...

Plus if not enough people report the issue, it will remain low priority...


----------



## NapalmV5

has nothing to do with windows/drivers/compatibility/etc

win7 win8.1 win10 all work fine going back to bios 5206 i didnt go further back and 5206 has no pcie lane/gen control whatsoever and all 4x titanx run at pcie3.0 no issues

5802 is the only bios that has port 2a link speed missing and runs slot 1 to pcie2.0 max regardless of os.. tested on win7 win8.1 and win10

im on 5701 and set all 4x 16x to gen3 just like on all the bioses that offer lane/gen control the only way to ensure pcie3.0 because not all bioses run automatically at pcie3.0


----------



## SRPlus

Quote:


> Originally Posted by *NapalmV5*
> 
> has nothing to do with windows/drivers/compatibility/etc
> 
> win7 win8.1 win10 all work fine going back to bios 5206 i didnt go further back and 5206 has no pcie lane/gen control whatsoever and all 4x titanx run at pcie3.0 no issues
> 
> 5802 is the only bios that has port 2a link speed missing and runs slot 1 to pcie2.0 max regardless of os.. tested on win7 win8.1 and win10
> 
> im on 5701 and set all 4x 16x to gen3 just like on all the bioses that offer lane/gen control the only way to ensure pcie3.0 because not all bioses run automatically at pcie3.0


I was running bios 5802 since its release November last year, and all four GPU's were at gen3 after.

Only after the Microsoft update containing Xeon and and x79 / C602 updates, the problem started. I know this because of my routine of clean installing the Nvidia drivers after SLI use before a big client project , for maximum stability, and re applying the gen3 patch. This was done the day before the Microsoft update, all showed gen3, after the update , primary only ran at gen2. I was surprised to see Xeon and x79 updates , so went though them at day, and they were related to pcie chipset devices. Didn't have a chance to fit the problem until now because of too much client work. Don't want to end up with a locked up workstation in the middle of a contract.

So it is definitely a compatibility problem with whatever Microsoft applied in the big December updates and the 5802 bios not having the linkspeed settings for the first slot. Older Bios like you said didn't even have the linkspeed options but the Gen3 patch would still work.

Thanks anyway NapalmV5, you took me down the right path, and going back to 5701 has fixed the Gen3 problem. If I get time I will take a closer look at the 5802 coding and see if I can spot the difference.

I will reply to asus and say the board has been fully functional before the updates and that its their bios that's the problem not a windows 10 problem.

Whatever Microsoft did, its looking for the gen3 for slot 1 in the bios probably though the auto configuration, not seeing it and revering to the default gen2 setting.

Quote:


> Originally Posted by *NapalmV5*
> 
> has nothing to do with windows/drivers/compatibility/etc
> 
> win7 win8.1 win10 all work fine going back to bios 5206 i didnt go further back and 5206 has no pcie lane/gen control whatsoever and all 4x titanx run at pcie3.0 no issues
> 
> 5802 is the only bios that has port 2a link speed missing and runs slot 1 to pcie2.0 max regardless of os.. tested on win7 win8.1 and win10
> 
> I'm on 5701 and set all 4x 16x to gen3 just like on all the bioses that offer lane/gen control the only way to ensure pcie3.0 because not all bioses run automatically at pcie3.0


Reverting back to Bios 5701 like NapalmV5 said cures the Gen3 problem for slot 1.

I'm not going to bother trying it, as I am a true believer in , if it ain't broke don't fix it, but I'm betting turning off auto configuration support bios and os also solves the problem.


----------



## koooowweeee

Rather than me starting a new thread i hope its ok o post my questuons in here as its sort of related.

Ive been given a pc. (No longer have one in sig.)

e5 2620 (2.0ghz) x2
asus z9pe d8
32gb 1600mhz corsair
Corsair h80i x2
Gtx590

I only use the pc for gaming so im wondering with games now using more cores and with the talk of DX12 using more cores as well should i keep this pc or sell it and buy something else with the cash. Will games use 2 cpus or will one be sat there wasted forever?

I will buy used so i should get something decent with caah?

Thanks for any help.

Esit. I will also be buy a new gpu in a month or 2


----------



## PLan2013

Any suggestions on the best deals for this board? I'm in the UK and about the cheapest price I can find in Europe is €440 -eBay prices in both the UK and US aren't particularly competitive .


----------



## SRPlus

Quote:


> Originally Posted by *PLan2013*
> 
> Any suggestions on the best deals for this board? I'm in the UK and about the cheapest price I can find in Europe is €440 -eBay prices in both the UK and US aren't particularly competitive .


I had a quick search around the usual places I buy from.

LambdaTek have the cheapest listing I can find for a new one, £386 inc VAT, but out of stock. I've purchased things from them before, with no issues, and very good customer service.

Next best I could find is Scan in the UK @ £419. I order about 60% of my hardware from them, and apart from having a few issues with delays to preorder items in the past, they have been good so far to deal with,

Hope that helps...


----------



## NapalmV5

Quote:


> Originally Posted by *SRPlus*
> 
> I was running bios 5802 since its release November last year, and all four GPU's were at gen3 after.
> 
> Only after the Microsoft update containing Xeon and and x79 / C602 updates, the problem started. I know this because of my routine of clean installing the Nvidia drivers after SLI use before a big client project , for maximum stability, and re applying the gen3 patch. This was done the day before the Microsoft update, all showed gen3, after the update , primary only ran at gen2. I was surprised to see Xeon and x79 updates , so went though them at day, and they were related to pcie chipset devices. Didn't have a chance to fit the problem until now because of too much client work. Don't want to end up with a locked up workstation in the middle of a contract.
> 
> So it is definitely a compatibility problem with whatever Microsoft applied in the big December updates and the 5802 bios not having the linkspeed settings for the first slot. Older Bios like you said didn't even have the linkspeed options but the Gen3 patch would still work.
> 
> Thanks anyway NapalmV5, you took me down the right path, and going back to 5701 has fixed the Gen3 problem. If I get time I will take a closer look at the 5802 coding and see if I can spot the difference.
> 
> I will reply to asus and say the board has been fully functional before the updates and that its their bios that's the problem not a windows 10 problem.
> 
> Whatever Microsoft did, its looking for the gen3 for slot 1 in the bios probably though the auto configuration, not seeing it and revering to the default gen2 setting.
> 
> Reverting back to Bios 5701 like NapalmV5 said cures the Gen3 problem for slot 1.
> 
> I'm not going to bother trying it, as I am a true believer in , if it ain't broke don't fix it, but I'm betting turning off auto configuration support bios and os also solves the problem.


i was on win10 builds prior and post december *when i tried bios 5802* and slot1 always @ pcie 2.0.. though ive never used gen3 patch.. maybe thats how yours ran at pcie 3.0

ive sent two complaints one for each z9pe-d8-ws i own including a pic of 5802 bios with port 2a link speed missing @

https://vip.asus.com/VIP2/Services/QuestionForm/TechQuery?lang=en-us

i hope 5802 wont be the last bios.. maybe if more us will file more complaints theyll release bios 590x sooner

ive held first position since july last year @ RealBench V2 Leaderboard

https://rog.asus.com/rog-pro/realbench-v2-leaderboard/

they dont have the resources to add nvme they cant ignore this motherboard its still relevant whether they like it or not

i will continue to stick it to their face









edit: got a proper response.. just what SRPlus got:
Quote:


> We apologize but we do not have the information on when will the new BIOS will be released.


going by how often a bios gets released we are looking at 2 more months at least.. lets hope its not 4 months from now


----------



## Roaches

Finally got this board all set up, Detects both E5-2670 Xeons on post, and just finished installing windows 7 and drivers.
Interestingly the board refuses to post with no display being stuck at B2 Post code when booting with a 290X2 Devil 13 installed. So I'm forced to stick with my older Nvidia card for the time being. I wonder if anyone else here had problems running current generation AMD cards?

So far I'm a huge happy camper. Unfortunately won't be able to truely utilize this spicy setup until the weekend to install some software.


----------



## SRPlus

Quote:


> Originally Posted by *NapalmV5*
> 
> i was on win10 builds prior and post december *when i tried bios 5802* and slot1 always @ pcie 2.0.. though ive never used gen3 patch.. maybe thats how yours ran at pcie 3.0
> 
> ive sent two complaints one for each z9pe-d8-ws i own including a pic of 5802 bios with port 2a link speed missing @
> 
> https://vip.asus.com/VIP2/Services/QuestionForm/TechQuery?lang=en-us
> 
> i hope 5802 wont be the last bios.. maybe if more us will file more complaints theyll release bios 590x sooner
> 
> ive held first position since july last year @ RealBench V2 Leaderboard
> 
> https://rog.asus.com/rog-pro/realbench-v2-leaderboard/
> 
> they dont have the resources to add nvme they cant ignore this motherboard its still relevant whether they like it or not
> 
> i will continue to stick it to their face
> 
> 
> 
> 
> 
> 
> 
> 
> 
> edit: got a proper response.. just what SRPlus got:
> going by how often a bios gets released we are looking at 2 more months at least.. lets hope its not 4 months from now


I've attacked Asus three times on this now.

So just have to wait I guess.

One guy I know did manage to get out of Asus that they are working on Win 10 drivers.

Had a go at that RealBench test.

Everything at stock speeds. Best score I could get was 171637.Didnt bother with the stress test.

That would put me at the top of Page two. Not bad I suppose for V1 E5-2600 Xeons and aging 2688 core TItans, not sure if multiple card configurations are tested though.

Had this message at the bottom "Unable to properly parse system specs. Please report in RealBench forum"

If I overclocked a little, would probably be able to get on 1st page, but don't see the point in potentially bricking my system used for work, for one test.


----------



## NapalmV5

edit: just checked my email and got a response to the last complaint.. they are working on it and will release asap

hey no problem doesnt matter the more z9 posted results the more z9 representin'









hmm strange never had a system specs sparse issue

yes opencl (luxmark) tests multiple cards.. though it doesnt make a big difference for the final score

saw your system and others' pics.. nice! i posted pics long time ago and since i dont have any at my profile heres mine nothing fancy just dusty


----------



## NapalmV5

Quote:


> Originally Posted by *Roaches*
> 
> Finally got this board all set up, Detects both E5-2670 Xeons on post, and just finished installing windows 7 and drivers.
> Interestingly the board refuses to post with no display being stuck at B2 Post code when booting with a 290X2 Devil 13 installed. So I'm forced to stick with my older Nvidia card for the time being. I wonder if anyone else here had problems running current generation AMD cards?
> 
> So far I'm a huge happy camper. Unfortunately won't be able to truely utilize this spicy setup until the weekend to install some software.


shouldve gone with ivy bridge if you want to stick with 290X2 Devil 13

even on ivy bridge all the 290s i tried/tested i was limited to 102 bus

what can be done on ivy bridge + titan x


----------



## Roaches

Embarrassingly turns out it was error on my end. I later figured out that one of my PCI-E cables was loose from the power supply when I installed my second 680 with the board post screen telling me one of my cards was not powered. Eventually I reinstalled one 290X2 and it posted perfectly to desktop, though since I'm not really gaming on this rig. I decided to move the 290X2s to another chassis with my former 4930K for that purpose.

Also the Force-Enable-Gen3 worked like a charm to get PCI-E slots to 3.0 from 2.0.

Did a test run on Cinebench, gave me a score of 1939 which seems about right in the same territory as other dual E5-2670 according to score database
http://cbscores.com/


----------



## NapalmV5

oh i thought it was incompatible since pcs 290s are on the gigabyte z68 ud7 b3 + 2600k

embarrassingly? the damn cables are plugged in perfectly and still no proper contact.. oh how many of those ive had!


----------



## Roaches

Good to know I'm not the only one alone with these small mistakes.. *Chuckles lightly*

As far as I know, recall reading something about those NF-200 chips not playing nice with the latest graphics hardware. Could explain why if some Z68 boards with those chips not working with the latest AMD hardware.

Cheers!


----------



## bitis

Hi guys,

Newbie on this forum.
So after reading up on the latest pages in this thread I would conclude that accepting Microsofts Win 10 update recommendation "What are you waiting for - update to Win 10" would NOT be a good idea yet - right?
Or, is it?

Oh, and I just updated bios to 5802. And realized that might not have been such a brilliant idea either...


----------



## Rob H

Please advise on adding second set of quad channel RAM.

I originally built my Z9PE back in 2013 with a single E5-2620 with the plans of upgrading eventually as I knew the Ivy versions would be out soon. I was quite satisfied with the relative performance from the single Xeon at the time and let time slide. Fast forward three years later and I finally was able to purchase a couple of ES E5-2695 V2's off Ebay. I thought the prices would drop a little sooner - oh well.

I originally purchased a Kingston 16 GB quad kit from the QVL (KINGSTON KHX2133C11D3K4/16GX) but now am unable to find a matching set for the second CPU that I will be installing. What RAM would be recommended for the second CPU that wouldn't cause compatibility issues? Would a new Kingston set with comparable clock & latency be ok? Or do I need to swap out my original ram for two new matching quad channel kits?

I realize that mixing RAM is not advisable but any suggestions would be greatly appreciated.

R


----------



## geort45

Quote:


> Originally Posted by *Rob H*
> 
> Please advise on adding second set of quad channel RAM.
> 
> I originally built my Z9PE back in 2013 with a single E5-2620 with the plans of upgrading eventually as I knew the Ivy versions would be out soon. I was quite satisfied with the relative performance from the single Xeon at the time and let time slide. Fast forward three years later and I finally was able to purchase a couple of ES E5-2695 V2's off Ebay. I thought the prices would drop a little sooner - oh well.
> 
> I originally purchased a Kingston 16 GB quad kit from the QVL (KINGSTON KHX2133C11D3K4/16GX) but now am unable to find a matching set for the second CPU that I will be installing. What RAM would be recommended for the second CPU that wouldn't cause compatibility issues? Would a new Kingston set with comparable clock & latency be ok? Or do I need to swap out my original ram for two new matching quad channel kits?
> 
> I realize that mixing RAM is not advisable but any suggestions would be greatly appreciated.
> 
> R


IMO they should run at the same speed so it should be AT LEAST the same Mhz. Also the voltage would need to be the same I think? I don't know if the Z9 lets you configure voltage independently for each CPU as the SR-2


----------



## Rob H

Quote:


> Originally Posted by *geort45*
> 
> IMO they should run at the same speed so it should be AT LEAST the same Mhz. Also the voltage would need to be the same I think? I don't know if the Z9 lets you configure voltage independently for each CPU as the SR-2


Thanks for the quick reply. I'll aim for a new comparable Kingston kit and purchase it from a local retailer with a good return policy.


----------



## geort45

Does anyone know if a 4800 V2 series Xeon would work in this mobo? They're both socket 2011, and Z9 supports V2 Ivy Bridge Xeons...

***corrected and added the V2, the V1 are socket 1567


----------



## geort45

Quote:


> Originally Posted by *geort45*
> 
> Does anyone know if a 4800 V2 series Xeon would work in this mobo? They're both socket 2011, and Z9 supports V2 Ivy Bridge Xeons...
> 
> ***corrected and added the V2, the V1 are socket 1567


Crap... seems 2011-1 socket variant isn't the same as 2011... which is referred to also as 2011-0 wth...


----------



## geort45

Finally bit the bullet and ordered 2 creamy 2670s! Cant wait!

Enviado desde mi iPhone utilizando Tapatalk


----------



## lowfat

Anyone ever able to successfully install ESXi?

I've had a Z9PE-D8 WS w/ 2 QA8Z (2660 ES) sitting pretty much unused since 2012. I want to replace my AMD Opteron 6176 / Supermicro H8SGL-8 w/ the Z9PE-D8 WS since it has way more PCIe expansion slots. But I'm running into an issue when I am trying to install ESXi 6.0.Coming up w/ the following error.
Quote:


> Decompressed MD5: 00000000000000000000000000000000
> 
> Fatal Error: 15 (Not Found)


----------



## Blindsay

Quote:


> Originally Posted by *lowfat*
> 
> Anyone ever able to successfully install ESXi?
> 
> I've had a Z9PE-D8 WS w/ 2 QA8Z (2660 ES) sitting pretty much unused since 2012. I want to replace my AMD Opteron 6176 / Supermicro H8SGL-8 w/ the Z9PE-D8 WS since it has way more PCIe expansion slots. But I'm running into an issue when I am trying to install ESXi 6.0.Coming up w/ the following error.


I had the same issue with 6.0, I dropped back to 5.5 and that worked.


----------



## lowfat

Quote:


> Originally Posted by *Blindsay*
> 
> I had the same issue with 6.0, I dropped back to 5.5 and that worked.


And it has been running fine w/o issue?

I couldn't get 5.5 to install either. However I just installed 6 on my Z170 machine then moved the USB stick to my Z9PE-D8 WS. It booted up fine no problem. Haven't moved any VMs over to it yet though.


----------



## Blindsay

Quote:


> Originally Posted by *lowfat*
> 
> And it has been running fine w/o issue?
> 
> I couldn't get 5.5 to install either. However I just installed 6 on my Z170 machine then moved the USB stick to my Z9PE-D8 WS. It booted up fine no problem. Haven't moved any VMs over to it yet though.


I ended up ditching it because I was having issues with gpu pass through (which I cant blame it because I was trying to do it with a consumer grade card) but other than that it was fine on 5.5


----------



## GendoIkari

I have a question! I've been considering updating my bios, because for some reason my USB keyboard isn't detected until windows starts. HOWEVER. I have six video cards, and I have to have the above 4GB memory accessible or some such setting turned on in the bios. How many cards would I have to pull before the machine would successfully boot so I could access the bios? I know I ran into this problem before, as Did Linus of Linustechtips, when he had 7 video cards on the newer version of this board. Also, would my raid arrays be ok? I'm using the onboard raid (Not the Marvell one thou.)


----------



## 3Dgeo

Hi there,

If someone willing to sell me any used Z9PE-D8 or any other dual 2011 socket MOBO contact me, I could buy even 2 of them if price is good.
No matter how old this message is - if you are selling one for good price - I'll buy it.
I'm looking for any manufacturer dual 2011 socket MOBO for good price, tho I prefer Z9PE-D8 WS.

Contact: djgramas(a)gmail.com


----------



## taitaitai

Hello everyone, I have a little problem with this motherboard:
I installed two Xeon E5 2670 processors and 64GB of ram corsair CMZ64GX3M8A1600C9 in bios I set this DDR3 1600MHz and functions regularly. But I have noticed that if I turn off the computer and restart, one of the banks is read only for 4GB. If the bios put the frequency drive and restart the ram is read at 1333MHz but with all 64GB. If I turn off and restart always I see all 64GB.
I set the latency to 9-9-9-24 1T and increased the voltage manually at 1,500 volts. But if pro to put (forcing 1600MHz) later I get back 60GB. I tested the RAM with various programs and the Rams have always passed the test.
In the motherboard I left not connected the EZ connector near the CPU 1 because I read that should be connected when present the second video card and serves to increase the stability of the tensions. Can someone help me understand the problem and find a solution. I have no experience with bi-processor boards. Thanks in advance.


----------



## Jann

I've had similar behaviour with my board and ram. It was showing as 40-48Gb instead of 64Gb if I forced the ram to 1600MHz.
Only after I found the exact timings (since CL 9 can be something like 9-9-10-10-24), and put those in, did the board run with 1600MHz and all 64Gb recognized correctly.
Before that I was just running it in auto 1333MHz and it worked fine too.

So either find the correct timing values, or just run it on 1333, it doesn't have any noticable impact.


----------



## Gdourado

Would I be losing much by going for a z9pa-d8 instead?
https://www.asus.com/Commercial-Servers-Workstations/Z9PAD8/overview/

I plan on dual 2670s but only a single GPU.

Cheers


----------



## Gdourado

Is the Z9PE-D8 WS compatible with 8GB Hynix 2Rx4 PC3-10600R DDR3 ECC Registered memory modules?


----------



## PLan2013

Problem ...

My machine runs for days without problem but sometimes randomly shuts down(like a power reset), tries to reboot, doesn't get very far, shuts down instantly again, typically goes into a loop of this. Next time I use it it may be fine again and run, non-stop, for days without issue.

I can't locate any kind of real indicator of the issue(have swapped out RAM), no logs, status code display on motherboard just seems to treat the issue as a reboot. It's apparently hardware related but what hardware is the cause?

I wondered about the power supply which is a Coolermaster GX Lite 700W PSU but apparently I'm only drawing 120W from the mains supply for my machine as I write this. I'm using a couple of 4-pin to 8-pin converters to split my two 4-pin CPU power leads if that matters?

I've no idea what the issue is ...


----------



## Gdourado

Quote:


> Originally Posted by *NapalmV5*
> 
> Quote:
> 
> 
> 
> Originally Posted by *Roaches*
> 
> Finally got this board all set up, Detects both E5-2670 Xeons on post, and just finished installing windows 7 and drivers.
> Interestingly the board refuses to post with no display being stuck at B2 Post code when booting with a 290X2 Devil 13 installed. So I'm forced to stick with my older Nvidia card for the time being. I wonder if anyone else here had problems running current generation AMD cards?
> 
> So far I'm a huge happy camper. Unfortunately won't be able to truely utilize this spicy setup until the weekend to install some software.
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> shouldve gone with ivy bridge if you want to stick with 290X2 Devil 13
> 
> even on ivy bridge all the 290s i tried/tested i was limited to 102 bus
> 
> what can be done on ivy bridge + titan x
Click to expand...

Can you please elaborate a bit more on this issue? I am thinking about getting this board and I have a 390x.

Cheers and thanks

Sent from my Nexus 6 using Tapatalk


----------



## PLan2013

Quote:


> Originally Posted by *PLan2013*
> 
> Problem ...
> 
> My machine runs for days without problem but sometimes randomly shuts down(like a power reset), tries to reboot, doesn't get very far, shuts down instantly again, typically goes into a loop of this. Next time I use it it may be fine again and run, non-stop, for days without issue.
> 
> I can't locate any kind of real indicator of the issue(have swapped out RAM), no logs, status code display on motherboard just seems to treat the issue as a reboot. It's apparently hardware related but what hardware is the cause?
> 
> I wondered about the power supply which is a Coolermaster GX Lite 700W PSU but apparently I'm only drawing 120W from the mains supply for my machine as I write this. I'm using a couple of 4-pin to 8-pin converters to split my two 4-pin CPU power leads if that matters?
> 
> I've no idea what the issue is ...


While swapping things around I may have located the issue ... by chance I moved a bunch of wires and noticed a rather thick, rigid, case front panel wire laying exactly on top off the motherboard power and reset button - pulled it away from the buttons and now my machine has booted and run perfectly for around 24 hours. Strange thing is the initial reboot issue happened randomly in between boot and actually running the Windows desktop for a while - it didn't seem consistent with a switch being held down permanently.

Effectively a freak occurrence if it is actually the fault ...


----------



## Gdourado

Will this memory work with the board and dual e5 2670?
http://www.ebay.co.uk/itm/Set-of-8-x-8GB-Hynix-2Rx4-PC3L-10600R-DDR3-ECC-Registered-HP-8GB-Total-64GB-/231904460700?hash=item35fe953f9c:g:uMcAAOSw2ENW8CIM&clk_rvr_id=1012039802244&rmvSB=true&ul_ref=http%253A%252F%252Frover.ebay.com%252Frover%252F1%252F710-53481-19255-0%252F1%253Ftype%253D4%2526campId%253D5337009344%2526toolId%253D10001%2526customId%253Dimt85o9tpx001u5200003%2526mpre%253Dhttp%25253A%25252F%25252Fwww.ebay.co.uk%25252Fitm%25252FSet-of-8-x-8GB-Hynix-2Rx4-PC3L-10600R-DDR3-ECC-Registered-HP-8GB-Total-64GB-%25252F231904460700%25253Fhash%25253Ditem35fe953f9c%25253Ag%25253AuMcAAOSw2ENW8CIM%2526srcrot%253D710-53481-19255-0%2526rvr_id%253D1012039802244


----------



## PLan2013

I bought this Hynix RAM a couple of weeks ago, seems to be running fine - http://www.ebay.co.uk/itm/381478525307 - Hynix is the world's second-largest memory chipmaker.


----------



## Struthio

Hi,

Can anyone help me with this ASPEED VGA ?

I shorted pins on MB for VGA (DIAG_VIEW1 position 1-2), but Windows nor Linux detect this device.
I have also external nVidia GPU, but I wanted to have two VGA adapters in system.


----------



## purduephotog

Pretty odd request here- but if you have a dead Z9PE-d8 and are interested in parting with it, I could use it for the parts to fix another one that went down.

Thanks much-


----------



## RedDingo81

Can anyone verify if 64GB of Corsair Dominator Platinum CMD32GX3M4A1600C9 will work ok in the Z9PE-D8 WS?
I'm currently building an all rounder (Gaming, Office, 3D rendering, Multimedia etc.) with 2x E5-2670's & not 100% sure on the Ram.

My Plan is the following list of components: http://www.overclock.net/lists/display/view/id/6496051

Any advise appreciated.

PS. I will not be water cooling any-more so I am using 2x H80i coolers instead, and the stock coolers on my GTX 690's (They've never even come close to overheating even at max recommended OC).


----------



## Lee Stevens

I bought this a week ago and since installing the CPU's (E5-2670) and GPU (MSI 980 TI) the system seems to get VERY hot idling around 70c for CPU which is crazy there also seems to be some hissing noise coming from it too (i think from CPU1). But it's not constant will do it then stop i'll do something it'll do it again. It's also rebooted whilst making the noise.

CPUs: E5-2670
Coolers: Noctua NH-U9DX
GPU: MSI 980 TI
PSU: is a AX1200i
RAM: 8x 8GB DDR3 1333 MHz PC3-10600 ECC)

The intel 602 chip is also very hot.

What could be causing everything to be running so hot?

EDIT: forgot to mention that CPU1 seems to also be the hotter out of the two.


----------



## rvb3n

Hi, I asked this sometime ago but I never got an answer so I hope now someone is able to help me: is it possible to install all the driver for this motherboard with Windows 10?

I tried again to do a fresh install but in Device Manager I keep having some unrecognized devices, how do you solve this?


----------



## Jann

Quote:


> Originally Posted by *rvb3n*
> 
> Hi, I asked this sometime ago but I never got an answer so I hope now someone is able to help me: is it possible to install all the driver for this motherboard with Windows 10?
> 
> I tried again to do a fresh install but in Device Manager I keep having some unrecognized devices, how do you solve this?


I had upgraded to Windows 10 from a 8.1 with all drivers installed, and all devices were recognised. You could also try sdi-tool to find the missing drivers.


----------



## rvb3n

Thanks for the answer Jann, but sdi-tool isn't able to find the missing drivers!


----------



## Jann

Quote:


> Originally Posted by *rvb3n*
> 
> Thanks for the answer Jann, but sdi-tool isn't able to find the missing drivers!


You could get the hardware id for the missing part in device manager, identify it in pcidatabase, and try installing the win8.1 driver for it.


----------



## yanman

Hi, forum newbie here. I am pretty certain I want to buy one of these.

Currently I have a board from a Lenovo D30 Thinkstation (eBay purchase) that turns out to be just too damn wide to fit in my Phanteks Enthoo Pro Case. I was set on modifying the case or just running without drive cages but that seems crazy and annoying so instead I think I'll just sell the board and change to this awesome looking Z9PE-D8

Anyone else running theirs in an Enthoo Pro?

One other question is around spacing between CPU socket centres. I'm running with 2 x Noctua U12S coolers which are basically right up against each other atm needing 125mm gap between socket centres.

Other components:
- 2 x Xeon E5-2670 (second-hand from eBay)
- 8 x 8GB Samsung Reg ECC DDR3 1600 (second-hand from eBay)
- Seasonic 1000 Platinum PSU
- Samsung 950 Pro 512GB M.2 (brand new, in a PCI-e lane adapter from eBay)
- Intel 120GB 320-series SATA SSD ... this will be a cache for ZFS
- USB 3.1 Type-C PCI-e card
- old Powercolor HD6970
- 4 x 3TB WD Greens... going to a ZFS zpool

[edit] Ordered one just now from amazon.de =D Got it for 275 Euros delivered to Aus!


----------



## Zoner

Quote:


> Originally Posted by *Donkey1514*
> 
> may I suggest modding the bios by adding the proper NVMe modules - http://www.overclock.net/t/1571271/tutorial-how-to-add-nvme-support-on-any-ami-uefi-bios-with-an-intel-chipset


Anyone actually get NVMe boot to work? I tried the link but got stuck on the bios validation protection on .Cap files


----------



## yanman

Quote:


> Originally Posted by *yanman*
> 
> (derp)


Crap I ordered the wrong board - a PA instead of PE... luckily managed to cancel


----------



## geort45

Quote:


> Originally Posted by *yanman*
> 
> Crap I ordered the wrong board - a PA instead of PE... luckily managed to cancel


PA? Looks more server oriented... good price though!


----------



## yanman

Quote:


> Originally Posted by *geort45*
> 
> PA? Looks more server oriented... good price though!


I'd need to add a USB controller, Audio.. and then I wouldn't get the benefit of a large community too :/

The intended use of this board is a VM host btw.. but the primary VM would be a Win10 one with a GPU in pass-through.


----------



## yanman

Quote:


> Originally Posted by *Gdourado*
> 
> Would I be losing much by going for a z9pa-d8 instead?
> https://www.asus.com/Commercial-Servers-Workstations/Z9PAD8/overview/
> 
> I plan on dual 2670s but only a single GPU.
> 
> Cheers


Hey did you end up going with the PA-D8 instead?


----------



## RushiMP

Lots of renewed interest in this platform with all the dirt cheap E5-2670 floating around.

Just managed to shoe horn this monstrosity into a Carbide Air 540. Had to drill and tap some CEB holes, but I think it worked out well.


----------



## MustangSVT

Where do you guys find this motherboard in stock? I've spent the past month looking for it in Canada with no success so far.


----------



## yanman

Quote:


> Originally Posted by *MustangSVT*
> 
> Where do you guys find this motherboard in stock? I've spent the past month looking for it in Canada with no success so far.


ComputerUniverse in Germany has some apparently and ships worldwide. Much better price than I've found here in Australia or on eBay elsewhere


----------



## MustangSVT

Quote:


> Originally Posted by *yanman*
> 
> ComputerUniverse in Germany has some apparently and ships worldwide. Much better price than I've found here in Australia or on eBay elsewhere


Cheers mate. The shipping to Canada seems very reasonable too. I also see they have the Z10PE-D8 WS in stock at a reasonable price.


----------



## Zoner

Quote:


> Originally Posted by *MustangSVT*
> 
> Cheers mate. The shipping to Canada seems very reasonable too. I also see they have the Z10PE-D8 WS in stock at a reasonable price.


Be careful Z10PE-D8 WS is a different 2011 socket then the Z9, it's not compatible with xeon E5-2600 series v1& v2


----------



## MustangSVT

Quote:


> Originally Posted by *Zoner*
> 
> Be careful Z10PE-D8 WS is a different 2011 socket then the Z9, it's not compatible with xeon E5-2600 series v1& v2


Right yeah, it's the newer v3/v4 CPUs, which I've seen some engineering samples on Ebay for cheap as well, but I suppose not as cheap as the v2 ones.


----------



## geort45

Quote:


> Originally Posted by *MustangSVT*
> 
> Right yeah, it's the newer v3/v4 CPUs, which I've seen some engineering samples on Ebay for cheap as well, but I suppose not as cheap as the v2 ones.


Does the Z10 run V4 Xeons then???

I just spotted an ES 2670 V3 on ebay, lowered its price, around USD 200, check it out!


----------



## MustangSVT

Quote:


> Originally Posted by *geort45*
> 
> Does the Z10 run V4 Xeons then???
> 
> I just spotted an ES 2670 V3 on ebay, lowered its price, around USD 200, check it out!


It is the same socket, so I assume so. Mind you, I don't know if it has the relevant BIOS update to support v4s. I saw a Xeon 2690 V4 ES for like 300 USD recently on Ebay, that woulda been a steal (14 cores, Turbo 3.1GHz).


----------



## geort45

Quote:


> Originally Posted by *MustangSVT*
> 
> It is the same socket, so I assume so. Mind you, I don't know if it has the relevant BIOS update to support v4s. I saw a Xeon 2690 V4 ES for like 300 USD recently on Ebay, that woulda been a steal (14 cores, Turbo 3.1GHz).


I checked on ASUS website and now its updated with V4 compatibility, however it needs BIOS 3204, I guess some Z10s on the market have an older BIOS so one would need another V3 CPU to update BIOS first...

$300 for the 14 cores... niiice!


----------



## Homerlovesbeer

Those 2670 V3/4 chips all over Ebay at the moment are ES chips, stolen engineering samples owned by Intel. None of them are the proper clock speeds either at 2.2Mhz (they should be 2.3Mhz) and bound to have a number of other issues.

I'm not tempted by them at the moment, especially when You'll need the Z10 board which costs a lot more than the Z9


----------



## Homerlovesbeer

Hi guys,

I've got a Z9PE-D8 board on its way along with 2 E5-2670 chips and 64Gb of Samsung ECC 1600Mhz Registered ram. My graphics card is an XFX R290x Double Dissipation card with 4Gb memory.

I'm putting it into a black Phanteks Enthoo Luxe case.

http://www.phanteks.com/Enthoo-Luxe.html


















Now onto cooling. I've never water cooled before but am keen to try it. I know that you can't overclock these processors much due to the locked multiplier so water cooling is pretty much overkill. HOWEVER I do wish to try it.

I was thinking of buying the EK-KIT X360 and buying another EK-Supremacy EVO water block for the second CPU. Will this work ok? https://www.ekwb.com/shop/ek-kit-x360










I've seen some loops that have the cooling going from one CPU straight into the second CPU. Wouldn't that mean one CPU was getting all the heat from the other? Here is a link and pics to explain.

http://www.xs-pc.com/showcase/ scroll down to the TCM Mod section.



















Does it matter that it is looped like this?

What would be the best air coolers to use in the Phanteks Entoo Luxe case? I know the Noctua coolers are very good (I have one in my current system, Q6600 lol) but I'm getting over the brown fan colour. It is starting to look pretty crap to me now and very dated.

I was thinking of getting a CoolerMaster Vanguard 1200W 80+ Platinum PSU in case I threw in another R290x for gaming. Will this be overkill? What would be an appropriate wattage PSU?

http://www.coolermaster.com/powersupply/enthusiast-v-series/v1200/

Cheers guys!

Homer


----------



## geort45

Quote:


> Originally Posted by *Homerlovesbeer*
> 
> Hi guys,
> 
> I've got a Z9PE-D8 board on its way along with 2 E5-2670 chips and 64Gb of Samsung ECC 1600Mhz Registered ram. My graphics card is an XFX R290x Double Dissipation card with 4Gb memory.
> 
> I'm putting it into a black Phanteks Enthoo Luxe case.
> 
> Now onto cooling. I've never water cooled before but am keen to try it. I know that you can't overclock these processors much due to the locked multiplier so water cooling is pretty much overkill. HOWEVER I do wish to try it.
> 
> I was thinking of buying the EK-KIT X360 and buying another EK-Supremacy EVO water block for the second CPU. Will this work ok? https://www.ekwb.com/shop/ek-kit-x360
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> I've seen some loops that have the cooling going from one CPU straight into the second CPU. Wouldn't that mean one CPU was getting all the heat from the other? Here is a link and pics to explain.
> 
> http://www.xs-pc.com/showcase/ scroll down to the TCM Mod section.
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> Does it matter that it is looped like this?
> 
> What would be the best air coolers to use in the Phanteks Entoo Luxe case? I know the Noctua coolers are very good (I have one in my current system, Q6600 lol) but I'm getting over the brown fan colour. It is starting to look pretty crap to me now and very dated.
> 
> I was thinking of getting a CoolerMaster Vanguard 1200W 80+ Platinum PSU in case I threw in another R290x for gaming. Will this be overkill? What would be an appropriate wattage PSU?
> 
> Cheers guys!
> 
> Homer


I'll just answer the temperature question for now: temp differences from loop order are negligible, the system is too small to allow the ambient to dissipate heat faster in one zone than another one. The temperatures will equalize in the whole loop.


----------



## geort45

Quote:


> Originally Posted by *Homerlovesbeer*
> 
> Those 2670 V3/4 chips all over Ebay at the moment are ES chips, stolen engineering samples owned by Intel. None of them are the proper clock speeds either at 2.2Mhz (they should be 2.3Mhz) and bound to have a number of other issues.
> 
> I'm not tempted by them at the moment, especially when You'll need the Z10 board which costs a lot more than the Z9


The board itself doesn't cost that much more, IMO the real Price difference apart from the processors themselves -which are for non-ES A LOT more expensive-, are the DDR4 prices, you can buy DDR3 8GB dimms super cheap now...


----------



## Homerlovesbeer

Thanks for the answer on the loops. Basically it doesn't matter?


----------



## Homerlovesbeer

My build....

Feedback welcomed. The water cooling really hammers the overall cost unfortunately.

http://au.pcpartpicker.com/list/wqqP6X


----------



## geort45

Quote:


> Originally Posted by *Homerlovesbeer*
> 
> Thanks for the answer on the loops. Basically it doesn't matter?


Sorry I didnt see your reply. Nah, better concentrate on having adequate airflow on your system to pour all that heat out. GL!


----------



## timh30

I've been given a ZPE-D8 WS motherboard, but it has some issues in that the Xeon CPU doesn't seem to be reported correctly in the BIOS or using apps such as CPU-Z or even the Intel CPU identifier utility (that reports it as too new/unknown - but recognises the number of cores and cache sizes). The board also only sees the first CPU, even when both are installed. The cpu's have been swapped (so both cpu's are good) and only the cpu in the first socket is ever seen. (I suspect the fact the cpu isn't identified correctly may be causing the issue with running both CPU's in SMP?)

I'm told the board and both cpu's were working previously, but after a BIOS update it stopped seeing the 2nd cpu. I have tried downgrading to much earlier versions of the BIOS, but updater utility will not let me roll back to the older 03xx versions.

It seems to run fine on just one cpu, but it would be nice to get it working properly with both sockets. Any ideas/suggestions as to what the problem might be and possible fixes?
Thanks.


----------



## Homerlovesbeer

Hi guys,

I just received my revision 1.4 board. Were there V1.1, V1.2 etc?

I want to build a new PC out of the box with a fresh install of Windows 10 using a Samsung 950 PRO 512GB M.2 card and a PCI express M.2 adaptor. I have read that you need to trick the board to boot from M.2 using a bios tweak, is that correct?

I'm following this guide....
http://www.overclock.net/t/1571271/tutorial-how-to-add-nvme-support-on-any-ami-uefi-bios-with-an-intel-chipset

What Bios should I upgrade to? Will I have the PCI Express slot drop back to 2.0 instead of 3.0 as it should be? I know some here have experienced these problems.

Anybody running windows 10 successfully without any problems?

Cheers,
Homer


----------



## shredhead666

Hi homer,

The only thing to boot off the drive is in bios under the boot tab go into csm menu and change pcie from legacy to uefi. Hope this helps


----------



## Homerlovesbeer

Thanks Shread,

So I can put my M.2 SSD drive in the slot, put in my windows 10 disk and install from scratch if I change the setting as you told me?

Cheers


----------



## shredhead666

Yea it should work. Also I don't know if you will have the same problem that I had with Windows 10 but if during the installation you get a blue screen of death then you might have to swap out your bios chip for a new one. If it installs without problem then you should be good to go


----------



## Homerlovesbeer

What the......

A windows 10 install crash ruins the bios! What type of shonky board is this!?

Go me worried now


----------



## shredhead666

No no lol it doesn't ruin the bios, what I meant it's that my bios didn't support some kind of power thing that Windows 8, 8.1 and 10 use. It was weird cuz Windows 7 installed fine but everything else would blue screen. So it was when I replaced them bios chip for a new one that all the os started working properly. Sorry for the confusion lol.


----------



## Homerlovesbeer

Lol oh ok I see now. So what bios version should I have before installing?

Cheers


----------



## shredhead666

The latest one from the Asus website, sorry I don't remember the number right now lol.


----------



## Zoner

I personally couldn't NVMe to work with stock BIOS so I made a modified one, you can try it if you want. I've included the last version of AFUDOS that had the /GAN command, the website removed all version with that command.

https://mega.nz/#!lYhT3Ibb!2flOAYfTfkzBDuvhUo8tfWlwfFKAU6OIDw3bSrDlkNQ

Make a DOS bootable USB drive.
Load files to USB and boot PC using it.
On DOS prompt type "AFUDOS.EXE modified.rom /GAN" command. /GAN is undocumented key meant to flash all blocks regardless to any software locks. If the output is like

Reading flash ............... done
Erasing All Block ............ done
Updating All Block ........... done
Verifying All Block .......... done

then your modified BIOS is flashed

Enter BIOS setup, on Boot tab enter CSM parameters it should look like this:
Launch CSM [Always]
Boot option filter [UEFI and Legacy]
Launch PXE OpROM policy [Legacy Only]
Launch Storage OpROM policy [UEFI Only]
Launch Video OpROM policy [Legacy Only]

Other PCI device ROM priorty [UEFI OpROM]

Save and exit. Follow SSD manufacturer's instructions on installing windows.


----------



## shredhead666

Hey zoner,

Just out of curiosity was the nvme drive you were trying to use a Samsung or Intel?

Thanks


----------



## Zoner

Quote:


> Originally Posted by *shredhead666*
> 
> Hey zoner,
> 
> Just out of curiosity was the nvme drive you were trying to use a Samsung or Intel?
> 
> Thanks


Intel 750 PCIe 400GB


----------



## shredhead666

Ah ok. I have a feeling that Intel has something in the firmware of their nvme ssds that makes them undetectable on older chipsets. I do not own one but know people have tried to get one working on older chipsets (x79,z77,z87) but it won't show in bios for booting. Whereas the 950 pro I can boot from (so far) any uefi bios lol.


----------



## Zoner

Quote:


> Originally Posted by *shredhead666*
> 
> Ah ok. I have a feeling that Intel has something in the firmware of their nvme ssds that makes them undetectable on older chipsets. I do not own one but know people have tried to get one working on older chipsets (x79,z77,z87) but it won't show in bios for booting. Whereas the 950 pro I can boot from (so far) any uefi bios lol.


I think the Samsung 950 Pro might be less picky about the BIOS, I did notice as I was stripping the encryption off Asus Z9PE-D8 WS latest BIOS it did not have the Modules named Nvme, NvmeSmm, NvmeInt13. I added them into my modded BIOS rom and the Intel 750 took it.


----------



## shredhead666

Yea it's Intel marketing bs. They want people to switch to the newer platforms to use the ssds lol. But nice job, I'm glad you got it to work


----------



## Migonacci

Hello Everyone,

I'm selling my pair of Xeon E5-2695 V2 ES 2.3 GHZ QD0y spec chips. I'm looking to upgrade so I don't need them anymore. PM me if interested.

Thanks.


----------



## Homerlovesbeer

Info on Asus regarding windows 10 BIOS.......
It seems they don't really have any plans for one :-(
Quote:


> Thank you for contacting ASUS Service Care.
> My name is Erik and it's my pleasure to help you with your problem.
> Thank you for bringing your issue into our attention. I am very sorry for any inconvenience this process may have caused.
> For your request, as per checking on the system, your product is not supported Windows 10. Therefore, we cannot provide you the Windows 10 BIOS update for this motherboard.
> I will transfer your request to the development team for further consider in the future and they will concentrate to release a new update for Windows 10 if it is possible. You may rest assured that all relevant information would be updated regularly on our website.
> Please kindly visit our website: https://www.asus.com/support for more detailed information.
> Sorry for this inconvenience that caused
> If you have any further questions, comments, or concerns please do not hesitate to let
> us know. We will be more than happy to assist you.
> Thanks!
> Wish you a nice day!


----------



## Gdourado

Just out of curiosity, in gaming with a discret gpu, like a 290X or gtx 980, how does a dual 2670 setup handle?
How does it compare to some low cost gaming builds like a bclk overclocked i3 6100 or even a AMD X4 880k?

Cheers!


----------



## Homerlovesbeer

Dunno bit I'll find out next week when my processors arrive as I have a 290x.


----------



## Homerlovesbeer

Tubing now run. I just ned to set up the drain at the bottom of the loop, install the CPU's and memory, run a leak test, then FINISHED!









I'm pretty happy with how it looks considering this is my first water cooled build


----------



## Zoner

Quote:


> Originally Posted by *Homerlovesbeer*
> 
> Info on Asus regarding windows 10 BIOS.......
> It seems they don't really have any plans for one :-(


I'm running windows 10 just fine, are you having install problems?


----------



## Homerlovesbeer

Nope. I haven't tried installing yet as I haven't installed the CPUs, memory or water cooling. Should be good for an install attempt in a couple of days.


----------



## renderz

Hi folks,

Just joined this forum, but I've been running my Z9 happily for several months now until just NOW.

I know this must have been asked before, but I couldn't find the answer:

When pressing the power button, the system tries to start but literally a second later it all shuts down, the POWER and RESET buttons are illuminated but don't function, and a bizzare little green light on the bottom right is lit.

Has this been resolved? Below are my specs:

2 x 2670
2 x 780
32gb non ECC ram (but like I said, this has been running sweet for months)
Corsair 1000W PSU
All watercooled

Please help!! It's my workstation so I need it to work asap.

Thanks.

My personal email address is [email protected]


----------



## RushiMP

No new hardware or reason to have an electrical fault right?

Have you tried resetting the CMOS?


----------



## renderz

Quote:


> Originally Posted by *RushiMP*
> 
> No new hardware or reason to have an electrical fault right?
> 
> Have you tried resetting the CMOS?


You are correct, no electrical fault.

And yes, I have reset the CMOS.

I have reseated the ram as well.


----------



## Zoner

Quote:


> Originally Posted by *renderz*
> 
> Hi folks,
> 
> Just joined this forum, but I've been running my Z9 happily for several months now until just NOW.
> 
> I know this must have been asked before, but I couldn't find the answer:
> 
> When pressing the power button, the system tries to start but literally a second later it all shuts down, the POWER and RESET buttons are illuminated but don't function, and a bizzare little green light on the bottom right is lit.
> 
> Has this been resolved? Below are my specs:
> 
> 2 x 2670
> 2 x 780
> 32gb non ECC ram (but like I said, this has been running sweet for months)
> Corsair 1000W PSU
> All watercooled
> 
> Please help!! It's my workstation so I need it to work asap.
> 
> Thanks.
> 
> My personal email address is [email protected]


My guess is a bad PSU, do you have an old one you can swap in to test it?


----------



## renderz

Quote:


> Originally Posted by *Zoner*
> 
> My guess is a bad PSU, do you have an old one you can swap in to test it?


Nope, I've spent the morning ripping it out and the power cables to attach to another PC, it works fine.


----------



## renderz

Hi folks,

I made a video in the hope some of you can help my issue. I'm desperate!!


----------



## Homerlovesbeer

I installed Windows 10 without modifying the bios. After installation and playing around with windows for a bit checking SSD speeds I installed a modified bios with the support for the native NVMe.

Bios options as follows;

Enter BIOS setup, on Boot tab enter CSM parameters it should look like this:
Launch CSM [Always]
Boot option filter [UEFI and Legacy]
Launch PXE OpROM policy [Legacy Only]
Launch Storage OpROM policy [UEFI Only]
Launch Video OpROM policy [Legacy Only]

Other PCI device ROM priorty [UEFI OpROM]

There is no discernable difference in speeds between CSM mode and UEFI modes.

Identical speeds. I did not reinstall windows 10 after modifying the bios for native NVMe support.

Would that make a difference?

Cheers,
Homer


----------



## Homerlovesbeer

Guys I'm really getting frustrated. My HD speeds are not great with sequencial read/writes however random read/write speeds are apalling! I have a Samsung 950 Pro M.2 SSD drive in PCI slot 3 using a Pci-e adaptor card. Random speeds are so bad I can't even be bothered finishing ATTO disk benchmark.

Bios is set as above. I wiped the first windows installation which was installed as Legacy with this one installed under UEFI.

My goodness, any suggestions on how to fix this?

Thanks!

EDIT: Problem SOLVED!

Device Manager - Disk Drives - NVMe Samsung SSD 950 - properties - policies

As this is not a SSD problem but NVMe driver policy behaviour.
The low performance was caused by the FUA(Forced Unit Access) command. It was originally blocked by the storage driver in MS Windows OS but the NVMe driver passes that command. And the benchmark tools that recently released invoke a bunch of FUA command so the performance of the SM951 NVMe dramatically goes down. MS is also aware of this issue and they are going to release a technical doc in MSDN near future. By that time, the way to avoid the FUA command is by checking "Turn off write-cache buffer" option (see below) under the device property of the SSD in your Device Manager".

So this is probably as good as it's going to get.

http://www.flexxmemory.co.uk/solid-state-drives-ssd/samsung-sm951-256gb-m-2-ngff-pcie-gen3-8gb-s-x4-solid-state-drive-ssd-2280-mzvpv256hdgl-00000-oem-nvme/


----------



## Homerlovesbeer

For the love of God, anybody who has the Asus Z9PE-D8 WS do not attempt to overclock it!









I made one change to the bios to the clock to 103 for a mild overclock and then spent the next 3 hours trying to get back into the bios!

I tried clearing the CMOS multiple times, no luck! Tried removing the CMOS battery then using the jumper pins to reset the CMOS, nothing!

I tried another graphics card to see if it simply wasn't getting a signal, nothing!

The ONLY thing that got it going again was to remove the second CPU, make sure the all the RAM except one DIMM for the 1st CPU was removed, move the graphics card to a PCIE slot used by the first CPU, disconnect all other devices such as HD's etc, then reset CMOS AGAIN!

Finally boot success into the BIOS!









I swear I was having a heart attack thinking my board was rooted or the BIOS corrupted. Holy hell, what a ***** to recover back into the BIOS, especially when water cooled and you need to remove the second CPU, clean up all the thermal paste, reapply and then refit.

Grrrr damn you ASUS


----------



## Zoner

Quote:


> Originally Posted by *Homerlovesbeer*
> 
> For the love of God, anybody who has the Asus Z9PE-D8 WS do not attempt to overclock it!
> 
> 
> 
> 
> 
> 
> 
> 
> 
> I made one change to the bios to the clock to 103 for a mild overclock and then spent the next 3 hours trying to get back into the bios!
> 
> I tried clearing the CMOS multiple times, no luck! Tried removing the CMOS battery then using the jumper pins to reset the CMOS, nothing!
> 
> I tried another graphics card to see if it simply wasn't getting a signal, nothing!
> 
> The ONLY thing that got it going again was to remove the second CPU, make sure the all the RAM except one DIMM for the 1st CPU was removed, move the graphics card to a PCIE slot used by the first CPU, disconnect all other devices such as HD's etc, then reset CMOS AGAIN!
> 
> Finally boot success into the BIOS!
> 
> 
> 
> 
> 
> 
> 
> 
> 
> I swear I was having a heart attack thinking my board was rooted or the BIOS corrupted. Holy hell, what a ***** to recover back into the BIOS, especially when water cooled and you need to remove the second CPU, clean up all the thermal paste, reapply and then refit.
> 
> Grrrr damn you ASUS


Quote:


> Originally Posted by *renderz*
> 
> Hi folks,
> 
> I made a video in the hope some of you can help my issue. I'm desperate!!


I wonder if this solution might also help renderz?


----------



## Wam7

Quote:


> Originally Posted by *Homerlovesbeer*
> 
> The ONLY thing that got it going again was to remove the second CPU, make sure the all the RAM except one DIMM for the 1st CPU was removed, move the graphics card to a PCIE slot used by the first CPU, disconnect all other devices such as HD's etc, then reset CMOS AGAIN!
> 
> Finally boot success into the BIOS!


Yeah, I read way, way back in the thread that some times the only the way is to re-seat a CPU. Saw a guy who tried to go to 105 and then spent hours trying to revive it, having to finally re-seat the CPU.


----------



## Homerlovesbeer

Quote:


> Originally Posted by *renderz*
> 
> Hi folks,
> 
> I made a video in the hope some of you can help my issue. I'm desperate!!


Hi mate,

Try what I did above. It may work however I'd say your board may have failed if you are not getting any post codes.

Best of luck!


----------



## renderz

UPDATE, some progress in finding out what's wrong with my board, but the results are not conclusive.. I still don't know if the board, or both CPUS and MOBO are faulty!!


----------



## Homerlovesbeer

Quote:


> Originally Posted by *renderz*
> 
> UPDATE, some progress in finding out what's wrong with my board, but the results are not conclusive.. I still don't know if the board, or both CPUS and MOBO are faulty!!


Did you try what solved my problems?


----------



## Homerlovesbeer

My build is now finished. The only thing I need to do now is install the LED's in the water blocks and pump.

It's looking pretty nice I think


----------



## Homerlovesbeer

Holy crap......

Am I reading your core voltage correctly at 1.9?









Have your CPU's survived this? How did you adjust the Bus speed to overclock? Wouldn't this effect the GPU as well?

Quote:


> Originally Posted by *Trekki*
> 
> Hello,
> 
> anyone experimented the Dual 2630v2 overclocking, or similar of these CPU.
> 
> At this time, i got 110mhz without freeze and problem, do you think its possible to get a 125 or 130mhz without damaged component?
> I put a little more Voltage for Vcore and VSA core (Vcore + 0.1) (VSA + 0.15)
> 
> Here my build :
> 
> MOBO = Z9PED8-WS (5103 bios)
> PSU = EVGA Supernova 1000G2 1000W 80PLUS Gold
> CPU(2x) = Intel® Xeon® Processor E5-2630 v2 (15M Cache, 2.60 GHz) (26 * 100mhz)
> AirC(2x) = Noctua NH-U12S LGA2011/115X AM2+ AM3+ FM1 FM2
> RAM(4x) = Kingston 16GB 240-PIN DDR3 ECC Registered DDR3 1600mhz
> GPU = EVGA GTX 780 SC 3 gb
> HardDrive = WD BLACK SERIES WD3003FZEX 3TB 64MB Hard Drive
> HardDrive = Kingston SSDNow V300 Series 240G 2.5" 240 GB SATA III
> 
> 
> Here an image of the 110mhz.
> 
> At this time,i have a 110mhz * 29 (with turbo at 31), this is possible to put that around 125 or 130mhz?
> 
> I tested 115 without freeze or bug, but i dont know how to fix my GPU, i dont want to OC more my GPU. And what is the best voltage for every thing in the box!
> 
> Anyone can help me?
> 
> Ty


----------



## spdaimon

I just got a one of these boards, and a second 2670. Getting q-code h2, anyone know what this means? Its not in the manual.

EDIT: Ok, The code is actually 62. The bottom LED on the 1st Q-code isn't working. Found an earlier post and moved one of the video cards, so at least I got video now.
Er..sorry, guess its actually b2.

EDIT2: The bottom LED on the 1st Q-code works intermittently. The second problem is that when I install the second CPU it just goes into a reboot loop. I put the second CPU in the old board I had, and it works fine, and I can get the first CPU to work on it ok. Using SR0KX E5-2670, and PNY DDR3-1866...hoping to get at least 1600Mhz but the RAM was almost the same price so I got the faster. BIOS is 5802.


----------



## linol

shredhead666, or anyone else running SAMSUNG 950 pro 512GB on our board? Hommerloves beer not included since he already done this! ;-)

What speed do you get in different benchmarks? Chrystal / Attos / as ssd? Screenshoots would be appreciated!


----------



## GendoIkari

Hey all, I just ran into an issue with my Windows 7 64bit Ultimate install. It's been pretty stable for the past 3 years or so, but since last month, after 4 hours uptime, I couldn't load any web pages or even **** down! Even restoring from a 2 week old backup didn't help..the issue came back 10 days after restoring.

So after pulling my hair out reinstalling drivers and my anti-virus software, I bit the bullet and "upgraded" to windows 10. It seems to handle my six video cards better, actually using each ones VRAM, but I have minor issue is that I cannot install the driver for the Intel C600 raid controller, which I do believe is the bottom 4 sata ports, which I'm not using anyway. I tried installing and it just told me to wait..for 40 mins. Luckily I made a system restore just before this point.


----------



## MustangSVT

Finally got my beast setup. Running two Titan X's at the moment. This build is meant for CUDA development and running many statistical models and machine learning models.


----------



## Homerlovesbeer

Cinebench of 1992 for my build.


----------



## palosanto

This thread is really big and although i've browsed a bit through it, I apologize if a solution for my problem has been posted before.
I've just finished building a workstation with this board and 2 Xeon 2670
This motherboard seems to be shipped with all the default jumper and bios setting set as a node rather than a standalone workstation. I think the majority of people buying this workstation board will not have any use for PXE, and remote functions, and will find the default settings for this motherboard annoying.
Afetr reading some posts in another forum I managed to avoid the annoying double or triple weird BIOS logo screens, disabling PXE etc.
Bios seem to detect properly the CPU's and RAM.

The following are the problems/questions that I have that perhaps I can solve with your help:

1.- My main problem is that BIOS won't detect ANY SATA devices. I've tried several hard drives in vaious SATA and SAS ports (for what I understand SATA HDD can be plugged in SAS ports and will work fine. Also tried different cables both for data and for power and I always get "no media detected" in the SATA Bios screen, they all show up as SATA 1: Not detected, SATA 2: Not detected, etc.
I tried changing the jumper that switches from LSI MegaRAID to RST (since I'm not using RAID) but it makes no difference.
None of the boot options show the hard drive either, only CD and floppy ...

2.- I realize that it's a complex and very advanced board but Is there a way to shorten or bypass the initial board tests or at least bypass the unnecessary ones since I'm using it as a regular workstation and not as a server? Right now it takes at least 90 seconds to get to the point where the HDD are not detected.

If any of you could share their optimal BIOS settings after their experience I would be very grateful.

My system:
BIOS 2604 (Latest for my CPU's)
Asus Z9PE-D8
2 x Xeon E5-2670 (from ebay $150 the pair)
Coolermaster Hyper 212 EVO
EVGA 750W gold PSU
Nvidia gForce 770 4gb
32 gb RAM 1866
256GB system SSD
3TB storage HDD

Below are some BIOS screens. The others have not been changed from default:






Windows 8.1 (if I ever manage to install it ....









Thanks in advance


----------



## Wam7

Quote:


> Originally Posted by *palosanto*
> 
> BIOS 2604 (Latest for my CPU's)


I'm not sure why you think 2604 is the latest for your CPU but 5802 is the latest for ALL CPU's. I have a 1.04 revision motherboard that came with 3206 so yours would seem to be pretty old though I don't think I see 2604 on the bios page. I strongly suggest you update to the latest 5802 using the inbuilt bios updater. Just format the USB stick as FAT32, drop the *.CAP file on it and your set to go.

I also have 2 x 2670's and they're running fine with Win10 Pro with bios 5802. I didn't actually do a fresh install as I just took the SSD with Win10 from another X58/Xeon 5670 build and just plugged it straight in. On first time boot it just downloaded a few drivers and then worked perfectly with just a couple drivers needing to be installed from the website.

EDIT. You have bios 5602 not 2604. I would still update and see how you get on.
Also I cured the start up bios problem of the Asus logo screen showing multiple times by changing *Force Bios* to *Keep Option ROM* which is a what I've done since Z68 days


----------



## palosanto

Thanks Wam7,

Where did you get the latest though ? When I go to Asus support page for this board, CPU support, this is the latest shown:



Would you mind sending me the CAP file or the exact link to download 5802 ?

I have my SSD also from another system with windows 8.1 and all the software. I've done that several times before preparing it with windows "Sysprep" and always worked fine. But the issue is that BIOS doesn't see any hard drives...
Anyways, I'll update BIOS first (if you can point me to the latest) . Maybe this solves the problem and BIOS sees my SSD.


----------



## Wam7

Quote:


> Originally Posted by *palosanto*
> 
> 
> 
> Would you mind sending me the CAP file or the exact link to download 5802 ?


You're on the wrong motherboard site, that's the Z9P*A*-D8 site. You can get it here.
https://www.asus.com/uk/Motherboards/Z9PED8_WS/HelpDesk_Download/


----------



## palosanto

I have actually the Z9P*A*-D8 ... I've been in the wrong forum all the time. I'm sorry.

Well that's even worse now. I cannot find a similar forum for this Mobo ....

EDIT: I found a comparison web page between the Z9PE and the Z9PA and they seem to vary only in form factor, integrated graphics (Z9PA has it), audio (Z9PA doesn't) and number of ports and PCI slots. Other than that they seem identical.
Is it a good idea to use the bios of one in the other ?


----------



## palosanto

SUCCESS:

It was a bad sata cable all along .... I changed the cable and bam ... I booted straight in windows, no hickups.
I quicly tested Cinebench 15 and I get a 2028 score !! My score with my other system (I7 4770k overclocked at 4.55Ghz was giving me 980 so this is awesome .... !

CPUz recognizes all cores and the memory running at 1866 mhz as I forced it in Bios.


----------



## nigdzie

Hey All,

best funny think with hit me today is ,,,, just swapped my 2xgtx980 to ROG STRIX-GTX1080-8G-GAMING and guess what happen ... no video output at all ....

Bios 5802
CPU 2xE5-2697V2

and big nothing, now all CMS option tested and no joy ...

There is any another crazy person like me who put ROG STRIX-GTX1080-8G-GAMING inside ?

Regards
_______________________
Edit

Self solved, all PCI-e cards out, then play one by one ... and on slot2 card was visible, then in bios CMS - uefi, and addon cards - uefi.

....

Solved

_______________________
Edit 2
Cine score


----------



## featheredtar

The ASMB6-iKVM is a chip that allows one to remotely restart/manage a workstation computer independently of the OS. I recently installed it in my Asus Z9PE-D8 WS mobo. After flashing the firmware to the chip, the Q-code diagnostic display on the mobo does an 80 second countdown procedure which allows the chip to initialize. When it finishes it stops at 0. Eventually the display changes to AE, which means "Legacy Boot Event". During this process nothing is outputted to the monitor.

If I remove the ASMB6-iKVM chip the mobo POSTs fine.

This is a pretty specific problem, but if anyone has any insight as to how I can get past the ASMB6-iKVM initialization process I'd appreciate it.


----------



## Wam7

Quote:


> Originally Posted by *featheredtar*
> 
> The ASMB6-iKVM is a chip that allows one to remotely restart/manage a workstation computer independently of the OS. I recently installed it in my Asus Z9PE-D8 WS mobo. After flashing the firmware to the chip, the Q-code diagnostic display on the mobo does an 80 second countdown procedure which allows the chip to initialize. When it finishes it stops at 0. Eventually the display changes to AE, which means "Legacy Boot Event". During this process nothing is outputted to the monitor.
> 
> If I remove the ASMB6-iKVM chip the mobo POSTs fine.
> 
> This is a pretty specific problem, but if anyone has any insight as to how I can get past the ASMB6-iKVM initialization process I'd appreciate it.


What bios version are you using?


----------



## spdaimon

Ok, I got it working! Got a second board from a local seller rather than from China as the first one was, and everything works perfectly! Like I thought it would! Sending the other one to ASUS for repair and may set it up in my NZXT Phantom 530.
Current build:
ASUS Z9PE-D8 WS
2x E5-2670 - ($60 ea. on fleabay now) My first one I bought a 8 months ago from same seller for over $100, that I used in a P9X79-E WS
2x CM Hyper 212 Evo
8x 8GB PNY Anarchy DDR3-1866 - (got for $26 ea at the Egg a few weeks ago) I can only get it to boot at 1600Mhz, which is fine. I was using Gskill Ares 1866Mhz and it was working at that speed on the
P9X79. The Anarchy DDR3-1600 was $1 less at that time.
2x R9 280X
EVGA 1000 GQ
Phanteks Enthoo Pro (in Titanium Green), added 3 Phanteks 140mm fans to the top, besides the front 200mm and back 140mm.

CPU temps are 53-55C for CPU1 and 65-69C for CPU2 full load,I think that is ok. Not sure how the gtx will affect the heat once those have a full load Using AC5 and tried not to smother it with TIM like I usually do. Guessing those temps are okay? I primarily want to use this to BOINC.

BTW, the Enthoo Pro fits the board perfectly except for 1 hole. I got some plastic snap-in supports with flat base online to fix that. Amazon didn't have them, found them via Google. I imagine the NZXT will have 2 or 3 holes that won't fit since it doesn't support the EEB form factor officially like the Phanteks did, and even then, 1 hole was missing.


----------



## featheredtar

I am using the latest BIOS. I have an update. Overnight the computer booted up!

Basically now whenever I restart it it takes 10-15 minutes of the error screen staying at 0 after the ASMB6-iKVM chip initializes, and then the mobo moves on to CSM initialization (diagnostic code 79). This process takes around 10-15 minutes. It all seems a bit excessive - what can I do to fix this?


----------



## geort45

Does anyone use a GTX 780 Ti with their Z9? I have a 780Ti that's giving me problems, it's in my SR-2, I want to be sure it's not related to that system's components. Last time I tried 780Ti on Z9 didn't work (and I upgraded BIOS), which BIOS version do you use?

Thanks,


----------



## kikkomaster

Hello, can anyone help im having problems getting gigabyte gtx 980ti G1 to boot on this motherboard.

First boot after bios reset gets me to bios, but as soon as i save and reboot it goes to boot loop, endlessly restarting itself and shutting down.

The card works flawlessly on my older computer, and so did my previous card on this board.

I have the newest bios flashed but still same problem occurs.

Any ideas?


----------



## TheBlademaster01

Which CPUs are you guys using?


----------



## kikkomaster

Hi, Im using a single E5-E2630 v2 in my setup.


----------



## kikkomaster

Quote:


> Originally Posted by *kikkomaster*
> 
> Hello, can anyone help im having problems getting gigabyte gtx 980ti G1 to boot on this motherboard.
> 
> First boot after bios reset gets me to bios, but as soon as i save and reboot it goes to boot loop, endlessly restarting itself and shutting down.
> 
> The card works flawlessly on my older computer, and so did my previous card on this board.
> 
> I have the newest bios flashed but still same problem occurs.
> 
> Any ideas?


Problem seems to be solved!

It was my PSU ( corsair hx 750i) cables that caused this weird issue, if i plugged both 8 pin cables like i normally do, it would not boot.

However gigabyte had a 8 pin to 2 x molex cable included in the package and after plugging that, and normal cable from psu it worked!

On my old rig with weaker psu it worked without the molex cable...that's very odd.

Anyway im very happy the problem is solved after hours of frustration


----------



## HiTekJeff

Hello,

I have ran into a major issue just today with the board not wanting to POST. There are no beeps and the POST code reads 64. However, I does not even get to the initial POST info about pressing a key to enter BIOS, initial part scan, memory, etc. It just hangs immediately after power up. I can confirm power is going to the board same as usual so it's not a PSU issue and the video card is powered on, at least the eVGA LED light is on.

Just wondering if anyone had advice on troubleshooting as it seems to be either a dead CMOS battery or video card despite power going to them as usual. I have the latest BIOS and everything was working just fine yesterday, just today it won't POST at all.

Thanks for any advice.


----------



## geort45

Quote:


> Originally Posted by *HiTekJeff*
> 
> Hello,
> 
> I have ran into a major issue just today with the board not wanting to POST. There are no beeps and the POST code reads 64. However, I does not even get to the initial POST info about pressing a key to enter BIOS, initial part scan, memory, etc. It just hangs immediately after power up. I can confirm power is going to the board same as usual so it's not a PSU issue and the video card is powered on, at least the eVGA LED light is on.
> 
> Just wondering if anyone had advice on troubleshooting as it seems to be either a dead CMOS battery or video card despite power going to them as usual. I have the latest BIOS and everything was working just fine yesterday, just today it won't POST at all.
> 
> Thanks for any advice.


I think the same happens to me when putting in a newer GPU


----------



## kolkata

Hi Guys,
Situation is strange but a friend of mine gave me a faulty Computer, with many great components, but it gets very often BSOD. The computer I used for few months, but i was getting angry at these random stuff, but I spent the last 2/3 days learning, learning and learning but it's hopeless. I was a total newbie and learnt so much to create boot things, change bios, etc etc.

Config :
RAM : Corsair 64GO 8x8
CPU 1 : Intel Xeon E5 2689 Sample
CPU2 : Intel Xeon E5 2689 Sample
Motherboard : Z9PE-D8 WS
GPU : Nvidia Geforce GTX 285
HDD : Samsung 250 Go SSD
cooling for CPUS : Z liquid cooling solutions

It shows Q-Code 66, so memory related and does MEMORY MANAGEMENT BSOD, or Kernel Power 41 63.

RAMS are coming from a working machine (

What I TRIED :

- I removed the GPU from slot 1 to slot 3
- I tried Memtest84+ with all situations I could : Trying on A1 each RAM : flawless
- I tried Memtest84+ with all situations I could : Only ONE CPU, TWO CPU, with 2, 4, 8 rams and all the time gets random bad red stuff happening.
- I changed the natural Speed of Mhz Ram ( which was 800 for one, and 667 for 7) to 1600, then manually 1600, or 1333
- I moved the natural speed of Mhz Ram to 10-10-10-24 and 2-T, or 1-T regarding the config.
- I S.M.A.R.T. all the hard drives
- I tried Intel processor Diagnostic Tool Test : Flawless for both CPUS
- I tried the Intel Extreme Tuning Utility : 5 minuts with memory : Failed / 5 minuts of processors : linpack stopped working.
- Changed the BIOS to the newest one.
- checked all the Ventilators to see.

Actually I have only one CPU with 2 rams on A1 B1, 1T, and it's pretty stable but still showing.

Any idea would help me a lot..

How can I troubleshoot more ?

VOLTAGES :


I was about to start putting the VSA a little bit more up, what are your advices guys ?


----------



## spdaimon

Things that come to mind are: Check the PSU perhaps? What model of PSU are using? Try a clean install of the OS? I'd try a clean install first, unless the PSU is under-powered. From your set up, I'd figure you'd need at least 650W.


----------



## kolkata

Hi Spdaimon,
Thanks for the reply.

I guess the PSU is not in cause, it's a be quiet! Dark Power Pro 1200W.

I'll try to install a new Windows 10 maybe on a new hard drive, let's see what happens..

On Windows 7, the 66 Code pops up here :


I tried to install windows 10 on an another hard drive, i Get error 22.

But as the memtest86+ at boot does errors, it's not really into the OS I guess..

ANOTHER THING IS, my RAM ( vengeance 8x8Go Kit ) is NOT ECC, maybe that's the problem !
I need maybe to try ECC Memory before...
I am with only one CPU now, i removed the second one.. but that's like this, it looks strange and does errors even with one PUC


----------



## spdaimon

22 showing on the Q-code? Thats normal. I'm using Windows 10, 64GB of PNY DDR3-1866 and I am getting random Memory Management or other memory errors myself it seems. I had it set to FORCE 1600..going to try it set to AUTO and see what happens. I haven't tried Memtest 86+ yet. I've tried turning off NUMA but that didn't seem to help. Working through many of the same problems you are.








ECC might help, but I don't think its necessary. If you are getting errors with Memtest 86+, then its not the OS. I missed that when you said it. I've tried looking at the memory voltages too, I just set them manually to 1.5v (stock voltage). Board reports 1.492 or something before.


----------



## Wam7

Quote:


> Originally Posted by *kolkata*
> 
> ANOTHER THING IS, my RAM ( vengeance 8x8Go Kit ) is NOT ECC, maybe that's the problem !
> I need maybe to try ECC Memory before...
> I am with only one CPU now, i removed the second one.. but that's like this, it looks strange and does errors even with one PUC


I suspect your memory is the problem. Random BSOD's can often be because of faulty memory. Test each stick on it own. Also run Memtest86+ on each stick by itself.


----------



## kolkata

Anyone has this : CORSAIR VENGEANCE 8x8Go Non ECC RAM ?
Ref : CMZ64GX3m8A16000C9
http://uplood.fr/visu.php?url=http:...s_13682465_10157174361775291_1851593691_o.jpg

I guess it's probably more the Sample Engineering CPUS..


----------



## Zoner

Quote:


> Originally Posted by *kolkata*
> 
> Anyone has this : CORSAIR VENGEANCE 8x8Go Non ECC RAM ?
> Ref : CMZ64GX3m8A16000C9
> http://uplood.fr/visu.php?url=http:...s_13682465_10157174361775291_1851593691_o.jpg
> 
> I guess it's probably more the Sample Engineering CPUS..


If you need some cheap EEC memory try here, I got my CPU and memory here
http://www.natex.us/Samsung-8Gb-M393B1K70DHO-CKO-64Gb-Kit-p/64gb-m393b1k70dho-cko.htm


----------



## Wam7

Quote:


> Originally Posted by *kolkata*
> 
> Anyone has this : CORSAIR VENGEANCE 8x8Go Non ECC RAM ?


Yes, I have a mixture of that RAM with some crucial memory and they both work fine together in my board.


----------



## Wam7

I have my Z9PE running with a pair of 2670's cooled by Noctua D14's and damn is this thing silent. Even with the fans at 50% both CPU's will only reach ~ 50c under full load. The other components are a passive GT720 GPU and a 256Mb Samsung EVO SSD. PSU is a 650W Corsair RMx (fans don't start spinning once you start to put the PSU under load and it never comes on).

My advice to anybody using theirs for x264 encoding is to disable HT. You will get better speeds this way.


----------



## spdaimon

I think my problem may be driver related. As soon as I put 64GB I get memory management error. Using Bluescreen View and a Microsoft KB, I found that the 2nd code relates to a corrupt PFT (page file table). I can run it 4 * 8GB, or 8 * 4GB without issue. I'm running Windows 10. Anyone else run into this issue? I've run Memtest86+ for 14 hours with no errors. Turrning off NUMA seems to slow down the errors. It seems to be random. It will happen while running BOINC, or could happen when I am just using the browser googling this problem.

EDIT: *sigh* Nope. Its definitely a bad memory stick. Stuck the other 1/2 of the memory in another rig and got Memory Management error with just 32GB.


----------



## kolkata

Hi,
Sorry to harass but I still get strange things going on with my MB.

The MB had been send back in march to ASUS in Czech republic. They said there were no errors.

I tried the MB with 2 Sample 2680 CPUS, and Corsair vengeance non ECC RAM. : Memory Management : ERROR 66

I bought retail 2670 with ECC REG RAM : Memory Management : ERROR 66

I tried with 2 pairs of CPUS, SAMPLE and RETAIL, and carious RAM, ECC, REG, not ECC, not ECC not REG.

Still the damn 66 code and erratic MANGEMENT MEMORY BSOD.

MOST strange, it looks that the CPU2 is sometimes like OVERheating compared to the first CPU.

Do you think I can get back the MB to ASUS ? I mean we never used this ******* computer because of this, and it looksmore and more that the Motherboard has a clear issue..


----------



## spdaimon

Quote:


> Originally Posted by *kolkata*
> 
> Hi,
> Sorry to harass but I still get strange things going on with my MB.
> 
> The MB had been send back in march to ASUS in Czech republic. They said there were no errors.
> 
> I tried the MB with 2 Sample 2680 CPUS, and Corsair vengeance non ECC RAM. : Memory Management : ERROR 66
> 
> I bought retail 2670 with ECC REG RAM : Memory Management : ERROR 66
> 
> I tried with 2 pairs of CPUS, SAMPLE and RETAIL, and carious RAM, ECC, REG, not ECC, not ECC not REG.
> 
> Still the damn 66 code and erratic MANGEMENT MEMORY BSOD.
> 
> MOST strange, it looks that the CPU2 is sometimes like OVERheating compared to the first CPU.
> 
> Do you think I can get back the MB to ASUS ? I mean we never used this ******* computer because of this, and it looksmore and more that the Motherboard has a clear issue..


Any bent pins? I think someone said maybe the heatsink is too tight and flexing the board. In my case, I found one of my brand new memory sticks was bad. I wasn't getting Memory Management :Error 66. Is this in Win10? I got error 1A and second code was like 0x14370 or something like that which said it was a corrupt PFT. I got another Z9PE-D8 that I sent to Asus because it refused to boot with 2 CPUs only 1. Not sure what thier repair warranty is..maybe 90 days? Best to talk to Asus about that. I know from experience with my Asus graphics card, I've had to send it back again. But that was only 1 out of 3 that I RMA'd. I bought faulty cards off eBay, its not me destroying them, I swear.


----------



## kolkata

Hi,
It looks like I reinstalled everything with erasing the win7 and putting windows 10 using windows 8 drivers.
I have Q-Code 22 when on Windows 10 but never had any BSOD as of now.. cross fingers...

BTW CPU-Z and HWinfo don't give me the same information about RAM. Which one shoudl I believe ?
CPUZ gives me the fact that the RAM is recognized as 667Mhz, as the HWinfo recognize at 800 with the correct timings...

Thanks alot!
Mathieu


----------



## kolkata

Did someone try a NH-D15 S pair on this mobo?
Thanks


----------



## kolkata

It works !


----------



## geort45

Quote:


> Originally Posted by *kolkata*
> 
> 
> 
> It works !


what dis you change


----------



## kolkata

Hi,

Honestly, it's strange.

I had many errors on Windows 7, it was getting impossible with these MEMORY MANAGEMENT stuff.
I did all I could, changing memory timings, the Voltages of CPU, RAM, etc, changed parts, change many RAMS, ECC, not ECC REG, NOT REG, changed slots PCI for stuff..
I bought a pair of retail 2670 instead of the ES 2680, honestly it didn't change anything.. I had the memory management too.

Since I am on Windows 10, everything is smooth. I just changed the second cooling tool for the 2nd CPU for a NH-D15S, which is very beautiful but also very efficient compared to the watercooling which was reacting erratic.. I am waiting money to buy the second one.
I have the Qcode 22, which looks normal.
I let the computer sleep for overnight then I had Qcode 30. I was starting to ask myself : ***, then it is normal, it's wake up from sleep mode.

Everything looks just right..

*ANYWAYS, I have a last question for you.*

I actually have a beast, with a Nvidia GTX 285 graphci card.
I do mainly Rhinoceros 5 3D, Vray rendering and Indesign/Photoshop Stuff.

*I would like to buy a used graphic card for around 200/300 euros, Which one could you recommend me for that kind of stuff to do ?*

Nvidia GTX 690 ?
Nvidia GTX 780 ?

Many thanks !
Mathieu


----------



## spdaimon

Yea, the Q-code is sometimes confusing. You'd think it wouldn't show anything if everything was okay. I have a P9X79-E WS board as well, it displays 'AA' I believe when its working okay.

I'm not sure what your local prices would be. For 200 euro, you could get a used GTX 960 or GTX 770 or on the AMD side you can R9 290 based on what I can find here in the US. Rhino 5 says it just requires a card with OpenGL 2.0 support which is pretty much any modern card.

You can fit 2 NH-D15S on this board? Hard to see in that photo. I am using two Cooler Master 212+ EVOs. I was getting 55C and 70C with the fans set to "Generic", which I guess thats is low or med. My case fans are also controlled by the mobo. I set them to HIGH now, and its come down to 52C on the first and 65C on the second. These temps are on full load on E5-2670s. Small improvement. Not bad for a $30 cooler. I'd be interested in getting slighty bigger heatsinks, but dont want to spend money to find that it won't fit. I am also using 2 R9 280Xs. I can see from the Wam7's photo, the NH-D15 will block the first PCIe slot.
Quote:


> Originally Posted by *kolkata*
> 
> Hi,
> 
> Honestly, it's strange.
> 
> I had many errors on Windows 7, it was getting impossible with these MEMORY MANAGEMENT stuff.
> I did all I could, changing memory timings, the Voltages of CPU, RAM, etc, changed parts, change many RAMS, ECC, not ECC REG, NOT REG, changed slots PCI for stuff..
> I bought a pair of retail 2670 instead of the ES 2680, honestly it didn't change anything.. I had the memory management too.
> 
> Since I am on Windows 10, everything is smooth. I just changed the second cooling tool for the 2nd CPU for a NH-D15S, which is very beautiful but also very efficient compared to the watercooling which was reacting erratic.. I am waiting money to buy the second one.
> I have the Qcode 22, which looks normal.
> I let the computer sleep for overnight then I had Qcode 30. I was starting to ask myself : ***, then it is normal, it's wake up from sleep mode.
> 
> Everything looks just right..
> 
> *ANYWAYS, I have a last question for you.*
> ts
> I actually have a beast, with a Nvidia GTX 285 graphci card.
> I do mainly Rhinoceros 5 3D, Vray rendering and Indesign/Photoshop Stuff.
> 
> *I would like to buy a used graphic card for around 200/300 euros, Which one could you recommend me for that kind of stuff to do ?*
> 
> Nvidia GTX 690 ?
> Nvidia GTX 780 ?
> 
> Many thanks !
> Mathieu


----------



## Wam7

Quote:


> Originally Posted by *spdaimon*
> 
> I can see from the Wam7's photo, the NH-D15 will block the first PCIe slot.


Mine are actually D14's which is slightly smaller than the D15. It only blocks the first PCI-E slot due to the orientation I have them in. I prefer the fans going from top to bottom rather than front to back. Though if I need the first PCI-E I would just use then front to back. I only have 1 GPU so have plenty slots so can afford to loose the first one.

The temperatures are ridiculously good.though. Like I said with the fans at 50% idle temps 35/36c and full load is just over 50c. Room temp is 24c. Thermal paste is the Noctua's own that comes with the D14.


----------



## spdaimon

Quote:


> Originally Posted by *Wam7*
> 
> Mine are actually D14's which is slightly smaller than the D15. It only blocks the first PCI-E slot due to the orientation I have them in. I prefer the fans going from top to bottom rather than front to back. Though if I need the first PCI-E I would just use then front to back. I only have 1 GPU so have plenty slots so can afford to loose the first one.
> 
> The temperatures are ridiculously good.though. Like I said with the fans at 50% idle temps 35/36c and full load is just over 50c. Room temp is 24c. Thermal paste is the Noctua's own that comes with the D14.


Ah, ok. I have 1 NH-D14 myself. It was cooling a 4930K on a X79 extreme 6 until I sent that board out for RMA. Played a bit musical cpus, lol! 4930K is now in the P9X79 and the E5-2670 from there is now in the Z9PE. I keep myself busy. Looking for a CPU now to test the RMA board once it comes back. Thinking a 4820K or maybe just another E5-2670 would be cheaper. Don't have a use for the board at the moment. Anyhoo, I was thinking using the D14 and getting another, but didn't know how well it would fit. Thats the reason I got the Extreme 6 in the first place, blocked the PCIe on the GA-X79-UP4 I have.


----------



## kolkata

I am 100% sure that both NH-D15S fit on a Z9PE-D8, for information. it's obvious. You just block the first PCI-E.


----------



## mapesh

can´t find the driverpage, is there no driversupport ?


----------



## geort45

Quote:


> Originally Posted by *mapesh*
> 
> can´t find the driverpage, is there no driversupport ?


it exists ive downliaded them


----------



## Wam7

Quote:


> Originally Posted by *geort45*
> 
> it exists ive downliaded them


It DID exist, it's not where it was when I downloaded the drivers. Have a look... no "Support" any more.


----------



## geort45

Quote:


> Originally Posted by *Wam7*
> 
> It DID exist, it's not where it was when I downloaded the drivers. Have a look... no "Support" any more.


Damn you're right... wth?? I did download drivers but for Win7, I don't know if your OS


----------



## geort45

Quote:


> Originally Posted by *Wam7*
> 
> It DID exist, it's not where it was when I downloaded the drivers. Have a look... no "Support" any more.


I'm very intrigued now, I'm planning to sell this system but in the meantime I will probably update to win10.

Check out this:

http://www.driverscape.com/manufacturers/asus/laptops-desktops/z9pe-d8-ws/2494

Tell me how it goes


----------



## spdaimon

I just installed Windows 10 on mine. Didn't have to download any drivers that I can remember. If there is something missing, I use something like Slimware's SlimDriver Free app to get anything missing. Win 8 and Win 10 are much better at having built-in or finding drivers than Win 7.


----------



## mapesh

here are the Bios Files http://filemare.com/de-de/browse/103.10.4.69/pub/ASUS/mb/LGA2011/Z9PE-D8-WS/BIOS


----------



## GendoIkari

Has anyone had issues with the drives attached to sata ports 1 and 2 over the years? I've had my system running since 2013 and suddenly it keeps dropping one drive or the other, and both are in raid 1 array via the onboard LSI embedded raid. I just replaced what I thought was a failed SSD, only to find it just dropped the old working one. And the old working one tests fine on other machines...


----------



## geort45

Quote:


> Originally Posted by *GendoIkari*
> 
> Has anyone had issues with the drives attached to sata ports 1 and 2 over the years? I've had my system running since 2013 and suddenly it keeps dropping one drive or the other, and both are in raid 1 array via the onboard LSI embedded raid. I just replaced what I thought was a failed SSD, only to find it just dropped the old working one. And the old working one tests fine on other machines...


PSU/PSU cable problems maybe?


----------



## Wam7

Or maybe the inconsistent nature of on-board RAID. One of the reasons I stopped using it years ago.


----------



## GendoIkari

Quote:


> Originally Posted by *geort45*
> 
> PSU/PSU cable problems maybe?


Quote:


> Originally Posted by *Wam7*
> 
> Or maybe the inconsistent nature of on-board RAID. One of the reasons I stopped using it years ago.


I already switched all cables, and I have two PSU's in this system. the other drive works fine if hooked up to one of the Marvell controller ports, and this just started happening in June of this year. it went from intermittent to consistent now.


----------



## SRPlus

Z9PE-D8 WS support drivers page is back up again. At the time of writing, no bios selection? But a backdated chipset update. Shows as October 2015, the last one I think has way back in 2013...

Disappointing, was hoping with the page being down a few days, that they were finally adding the Win 10 support drivers...

They did originally promise when Win 10 was launched that all x79 based motherboards will be getting win 10 updates and support, but looks like its never going to happen for this Mobo.

I'll give it a few days before trying out the chipset drivers, looks like there's an audio update too. Give Asus time to sort out the support page. Sometimes they do put up and then take down drivers again... Looks incomplete ATM so will wait...


----------



## Jirkas

Hi friends,

I would like to ask for this MOBO. I have read this discussion and I want to make a decision, if I should buy it or not.

I would like to build a new PC for following software:

- Photoshop CS6
- Premiere CS6
- After Effects CS6
- Hitfilm 3
- Davinci Resolve

(SW, that I want to use in Future too
- Elements 3D
- Blender

Now I am using i7 3770, 16 GB RAM, Nvidia GTX Titan (6GB), 240GB SSD(Sata3), 4TB HDD(Sata3). I edit videos, but in future I would like to make more After Effects CS6 and 3D things.

I have seen, that the 2011 CPUs are now cheap and I think, that dual Xeon 2011 is maybe a good idea. I would like to build:

- Z9PE-D8 WS
- Dual Xeon 2670 v1
- Nvidia GTX 1080 (1x or 2x) or Nvidia GTX 980 (1x or 2x) (I can start with my GTX Titan).
- X GB RAM
- Win 7 (or Win 10)

The questiion is, if it is possible. Are the GTX 980 or GTX 1080 compatible with this motherboard? I am asking, because I have read this Z9PE-D8 WS topic and I find this MOBO very problematic: Bios updates, Graphic cards compatibility, RAM compatibility ...

What do you think about it? Thank you very much for your help ...


----------



## SRPlus

Quote:


> Originally Posted by *Jirkas*
> 
> Hi friends,
> 
> I would like to ask for this MOBO. I have read this discussion and I want to make a decision, if I should buy it or not.
> 
> I would like to build a new PC for following software:
> 
> - Photoshop CS6
> - Premiere CS6
> - After Effects CS6
> - Hitfilm 3
> - Davinci Resolve
> 
> (SW, that I want to use in Future too
> - Elements 3D
> - Blender
> 
> Now I am using i7 3770, 16 GB RAM, Nvidia GTX Titan (6GB), 240GB SSD(Sata3), 4TB HDD(Sata3). I edit videos, but in future I would like to make more After Effects CS6 and 3D things.
> 
> I have seen, that the 2011 CPUs are now cheap and I think, that dual Xeon 2011 is maybe a good idea. I would like to build:
> 
> - Z9PE-D8 WS
> - Dual Xeon 2670 v1
> - Nvidia GTX 1080 (1x or 2x) or Nvidia GTX 980 (1x or 2x) (I can start with my GTX Titan).
> - X GB RAM
> - Win 7 (or Win 10)
> 
> The questiion is, if it is possible. Are the GTX 980 or GTX 1080 compatible with this motherboard? I am asking, because I have read this Z9PE-D8 WS topic and I find this MOBO very problematic: Bios updates, Graphic cards compatibility, RAM compatibility ...
> 
> What do you think about it? Thank you very much for your help ...


I know someone using 3x GTX Titan ( Maxwell ) on this MoBo without issues.

I don't see why Pascal should have any problems on it.

I think the general rule with this motherboard is dual Xeons = Registered ECC ram

If you are planning to go for the GTX 1080, make sure the app's you use have Pascal compatibility first.

I use Iray in 3ds max and its still waiting for Pascal support, but it should be available very soon according to the developer site.

Are you buying new or second hand components?

You also have to consider that the X79 platform is pretty much EOL ( end of life ) and any future support, drivers etc, is very much back of the queue for developers. You mentioned a few games, and SLI works fine if you plan to have more than one GPU, but if you want surround, its a no go on this mobo..

Even the newer 2011- 3 platform is nearing its EOL as far as future CPU's go. The next Xeon refresh will be Purely as far as I know which will require the LGA 3647 socket.

But if you can pick the components up cheap, the Z9PE-D8 is still a powerful platform to build from.


----------



## Jirkas

Components:

Motherboard - new

CPUs - second hand

GTX 1080 - new , GTX 980 - new or second hand, GTX Titan - I have already 1

RAM - new or second hand (for start I would like to use my unreg. modules)

HDDs - new

Case - new https://www.tsbohemia.cz/nzxt-phantom-530-bila_d205577.html?fulltextword=eatx

Software:

After Effects CS6 - Program support GTX cards until Kepler core , with workaround it is possible to use Maxwell core too, I think, that with this workaround it is possible to use Pascal too

Premiere CS6 - Maxwell works, Pascal I don't know ...

Elements 3D - support OK

Davinci Resolve - free updates

By Adobe Products I want to stay by CS6 version - I have to pay nothing per month. For this Reason I have no problem with old platform. I am hobby filmaker only ...


----------



## SRPlus

Quote:


> Originally Posted by *Jirkas*
> 
> Components:
> 
> Motherboard - new
> 
> CPUs - second hand
> 
> GTX 1080 - new , GTX 980 - new or second hand, GTX Titan - I have already 1
> 
> RAM - new or second hand (for start I would like to use my unreg. modules)
> 
> HDDs - new
> 
> Case - new https://www.tsbohemia.cz/nzxt-phantom-530-bila_d205577.html?fulltextword=eatx
> 
> Software:
> 
> After Effects CS6 - Program support GTX cards until Kepler core , with workaround it is possible to use Maxwell core too, I think, that with this workaround it is possible to use Pascal too
> 
> Premiere CS6 - Maxwell works, Pascal I don't know ...
> 
> Elements 3D - support OK
> 
> Davinci Resolve - free updates
> 
> By Adobe Products I want to stay by CS6 version - I have to pay nothing per month. For this Reason I have no problem with old platform. I am hobby filmaker only ...


Asus have some tried and tested non ECC unregistered ram on the support page for this Mobo.

http://dlcdnet.asus.com/pub/ASUS/server/ESC2000_G2/Manual&QVL/ESC2000_G2_Z9PE-D8_WS_AVL_UDIMM_20160219.pdf?_ga=1.59924397.1491690690.1472474722

But I am sure there are some members of this forum that can recommend some tried and tested ram not on that list.

This Mobo has been virtually hassle free for me over the last 4 years. apart from the odd problem caused by windows update, or bios not fit for release, but those problems can happen on any system and are easily solved. Its bios is quite simple and I've run both legacy and UEFI on this system without problems. Legacy on Win 7 pro x64, and UEFI on Win 10 Pro x64.

Always read though the manual, and make sure that both CPU 8-pin power sockets are connected to the PSU.

Blue Pci-e interface are x16, black Pci-e x8. CPU 0 controls slots 1 to 4, CPU 1 controls slots 5 to 7.

I use the two blue Sata 6gb connectors for my main O/S and app SSD, and projects SSD, and the marvel 6gb headers for mass storage and Optical. The old Sata ports are not in use.

Once your system is built, if you run into any issues I'm sure someone here can talk you though it.

Good Luck 

oh and you may want to drop Asus support an email just to ask on Pascal compatibility, unless someone on this thread has already tested this...

The only downside to owning an Asus motherboard is Asus support unfortunately.....


----------



## Jirkas

I have read , that many people had a problem with the Graphic cards. The motherboard did not recognize the graphic cards and the users had to update BIOS etc. How can I update bios, when my graphic card does not work and I see nothing ?! For this reason I am not sure, if GTX 1080 run. Asus does not make any new update







((.


----------



## Jirkas

I have checked Z10PE-D8 WS and X99s too, but the CPUs are always expensive ...


----------



## geort45

I wasnt able to use my 780Ti but I didnt insist. I use ECC ram is super cheap and I use 8GB dimms


----------



## Jirkas

Does the motherboard have integrate graphic card? Sometimes I see integrate graphic card in description ?! I see no VIDEO OUTPUT on the pictures ...


----------



## SRPlus

Quote:


> Originally Posted by *geort45*
> 
> I wasnt able to use my 780Ti but I didnt insist. I use ECC ram is super cheap and I use 8GB dimms


I'm using 4x old GTX titans ( Kepler ) without problems. So the 780ti should also work. I went back to Bios 5701 as the newest bios has a problem with the first slot running at Gen 3.

Strange how temperamental this mobo is, seems like its a lottery getting a good one which isn't very encouraging for Jirkas, I got lucky I guess, but from what I know the SR-2 you are using is also riddled with problems, and EVGA development and support gave up on it a few years back, as it was too much of a headache.

I rarely use CPU's for rendering anymore, so my next build will probably be a single socket. the X99-e WS maybe, with 4 x Titan X ( Pascal) or maybe I will wait until this time next year and build on the new platform. Hopefully there will be a fully loaded Pascal by then...

Jirkas

The Z10 and X99 are expensive plus the CPU's, but they may offer a more future proof platform, especially compatibility wise.

Have you cross checked on somewhere like CPU Monkey to see which V3 or V4 Xeons may offer the same processing power as the old 2670? I haven't checked but the lower end V3/V4 Xeons may match the power of the old 2670's but cheaper.


----------



## SRPlus

It has a basic VGA controller built in which is disabled on later bios versions. The VGA connector is on a PCI slot plate in the box and connects to a motherboard header if you want to use it, but I doubt anyone does, mine is still unopened in the box in the loft.


----------



## spdaimon

Quote:


> Originally Posted by *Jirkas*
> 
> Does the motherboard have integrate graphic card? Sometimes I see integrate graphic card in description ?! I see no VIDEO OUTPUT on the pictures ...


It does, there is a VGA header on board that you would need the accessory bracket for. Its not on the I/O back ports.


----------



## geort45

Quote:


> Originally Posted by *SRPlus*
> 
> I'm using 4x old GTX titans ( Kepler ) without problems. So the 780ti should also work. I went back to Bios 5701 as the newest bios has a problem with the first slot running at Gen 3.
> 
> Strange how temperamental this mobo is, seems like its a lottery getting a good one which isn't very encouraging for Jirkas, I got lucky I guess, but from what I know the SR-2 you are using is also riddled with problems, and EVGA development and support gave up on it a few years back, as it was too much of a headache.
> 
> I rarely use CPU's for rendering anymore, so my next build will probably be a single socket. the X99-e WS maybe, with 4 x Titan X ( Pascal) or maybe I will wait until this time next year and build on the new platform. Hopefully there will be a fully loaded Pascal by then...
> 
> Jirkas
> 
> The Z10 and X99 are expensive plus the CPU's, but they may offer a more future proof platform, especially compatibility wise.
> 
> Have you cross checked on somewhere like CPU Monkey to see which V3 or V4 Xeons may offer the same processing power as the old 2670? I haven't checked but the lower end V3/V4 Xeons may match the power of the old 2670's but cheaper.


Yes, SR-2 was a great motherboard but it was troublesome for many. It runs my 780 Ti without problems though. It had problems detecting some memory configurations too, but it's perfectly fine with 32GB ATM.

I dunno what BIOS I tested on the Z9 for the 780, I'll try with 5701 as you suggest, I did use the first slot too as you mention.

For future-proofing, one might wanna go get a Z10... Xeon V2's are more expensive but 2670's V2 can be found at affordable prices, V3 are off the charts mostly.


----------



## Jirkas

https://www.cpubenchmark.net/high_end_cpus.html

1x Intel Xeon E5-2670 @ 2.60GHz
12,419 PassMark points
100 Eur Ebay

1x E5-2630 v3
12797 PassMark points
700 Eur Ebay

Intel Core i7-6900K
17574 PassMark points
1000 Eur NEU

https://www.cpubenchmark.net/multi_cpu.html

2x Intel Xeon E5-2670 @ 2.60GHz
18,562 PassMark points
2x100 Eur Ebay

2x E5-2630 v3
18,688 PassMark points
2x700 Eur Ebay


----------



## geort45

Nice comparison. Yup v3 are expensive as hell, and thats a 2630 not a 2670 which is 10 or 12 core I think, so just imagine the price haha


----------



## Jirkas

To SRPlus
Quote:


> I rarely use CPU's for rendering anymore, so my next build will probably be a single socket. the X99-e WS maybe, with 4 x Titan X ( Pascal) or maybe I will wait until this time next year and build on the new platform. Hopefully there will be a fully loaded Pascal by then...


If you buy a new PC, I can buy in Future your old PC







.


----------



## Jirkas

I don't need urgently this PC, I can wait - I am hobby filmmaker only. Now I have i7 3770K, 16GB RAM, GTX Titan (Kepler), Win 10. But I have seen the prices of old xeons i.e. 2670 v1 and I have got this Idea. I would like to buy Elements 3D and I found good to have a new PC for this software ...


----------



## Jirkas

Quote:


> Originally Posted by *SRPlus*
> 
> It has a basic VGA controller built in which is disabled on later bios versions. The VGA connector is on a PCI slot plate in the box and connects to a motherboard header if you want to use it, but I doubt anyone does, mine is still unopened in the box in the loft.


I love Asus support. I have call Asus hotline 2x and the asus people have told me, that this motherboard is without graphic chip - When my external graphic card does not work, I have no chance with this motherboard something to do ...


----------



## Jirkas

2x 110 Eur CPUs
1x 450 Eur motherboard
2x 80 Eur Coolers
1x 100 Eur Case
1x 150 Eur PSU

1080 Eur
+RAM
(+ Graphic Cards etc)


----------



## SRPlus

If you are like me and have lots of spare components laying around, you could do what I did.

In may 2012 I started off with the Z9PE-D8 WS, one E5-2687w, and 32GB ( 4x8GB ) of DDR4 quad channel ECC ram. All new.

I used an old Xigmatek Elysium case, an old Dark Power Pro 1200W PSU from an old build and an old GTX 580 3GB, old 256GB SSD, and old HDD's, and old Akasa Viper fans from an old build.

Around August 2013 I replaced my GTX 580 with 2x GTX Titans, and January 2014 I purchased the second CPU, and another 32GB ram. 4 months later I purchased the other 2x GTX Titans off ebay for a vewwyy sick price  , replaced my power supply and changed my case. Only a few changes since then, new 1TB SSD, better fans, but I built the system up slowly, there was no way at the time I could afford to build it in one go.


----------



## geort45

Quote:


> Originally Posted by *SRPlus*
> 
> If you are like me and have lots of spare components laying around, you could do what I did.
> 
> In may 2012 I started off with the Z9PE-D8 WS, one E5-2687w, and 32GB ( 4x8GB ) of DDR4 quad channel ECC ram. All new.
> 
> I used an old Xigmatek Elysium case, an old Dark Power Pro 1200W PSU from an old build and an old GTX 580 3GB, old 256GB SSD, and old HDD's, and old Akasa Viper fans from an old build.
> 
> Around August 2013 I replaced my GTX 580 with 2x GTX Titans, and January 2014 I purchased the second CPU, and another 32GB ram. 4 months later I purchased the other 2x GTX Titans off ebay for a vewwyy sick price  , replaced my power supply and changed my case. Only a few changes since then, new 1TB SSD, better fans, but I built the system up slowly, there was no way at the time I could afford to build it in one go.


Except that it's DDR3


----------



## geort45

Quote:


> Originally Posted by *Jirkas*
> 
> I love Asus support. I have call Asus hotline 2x and the asus people have told me, that this motherboard is without graphic chip - When my external graphic card does not work, I have no chance with this motherboard something to do ...


It has onboard video,wth


----------



## SRPlus

Quote:


> Originally Posted by *geort45*
> 
> Except that it's DDR3


Haha yep, my system wouldn't have worked very well the last 4 years if I got that wrong 

I've been looking at X99 and C612 motherboards the last few days, deciding whether or not to upgrade now or wait, had DDR4 stuck in my head :- /


----------



## SRPlus

I'm in two minds about upgrading though, driving me crazy, got the savings, itching to burn haha...

I think its better to upgrade at the start of a new platform than at the end of one, and the 2011-3 Z10PE-D8 WS and X99-E WS have had their last CPU refresh as far as Xeons are concerned... Xeon V5's will need a new socket.

Plus with pci-e 4, NV-link server platforms, and the Xeon Purley's on the very near horizon, I'd feel a bit gutted forking out on a new workstation, being stuck on a 2011-3 platform and not being able to upgrade to the latest and greatest in the future.

My current system is ok, it does the job I built it for well, so although I'm itching to upgrade to the current Xeons, and 4x Titan Pascal, to cut my render times down to a third, It looks like I will be playing the waiting game again......

I decided not to upgrade to the 2011-3, decided not to upgrade to 4x GTX Titan X's ( Maxwell ) because the 46% performance increase over my old Titans rendering in Iray was not worth the £4K price-tag, but the newer Titan X would definitely be worth the upgrade, as it would cut my renders to a third, and pay for themselves in a few months, but there is a very high probability that we will be seeing a fully loaded GP102 next year..

Anyone else stuck in the in the upgrade now or wait feedback loop?


----------



## geort45

Quote:


> Originally Posted by *SRPlus*
> 
> I'm in two minds about upgrading though, driving me crazy, got the savings, itching to burn haha...
> 
> I think its better to upgrade at the start of a new platform than at the end of one, and the 2011-3 Z10PE-D8 WS and X99-E WS have had their last CPU refresh as far as Xeons are concerned... Xeon V5's will need a new socket.
> 
> Plus with pci-e 4, NV-link server platforms, and the Xeon Purley's on the very near horizon, I'd feel a bit gutted forking out on a new workstation, being stuck on a 2011-3 platform and not being able to upgrade to the latest and greatest in the future.
> 
> My current system is ok, it does the job I built it for well, so although I'm itching to upgrade to the current Xeons, and 4x Titan Pascal, to cut my render times down to a third, It looks like I will be playing the waiting game again......
> 
> I decided not to upgrade to the 2011-3, decided not to upgrade to 4x GTX Titan X's ( Maxwell ) because the 46% performance increase over my old Titans rendering in Iray was not worth the £4K price-tag, but the newer Titan X would definitely be worth the upgrade, as it would cut my renders to a third, and pay for themselves in a few months, but there is a very high probability that we will be seeing a fully loaded GP102 next year..
> 
> Anyone else stuck in the in the upgrade now or wait feedback loop?


If you have the money to buy the newest Xeon motherboard and CPU when they come out, go for it. I just wonder what the prices will be, a basic V3 is very expensive, just imagine a V5.

This is a game you can't always be winning, you can have a year of "being at the top" at most, and only if you get the newest platform as it comes out. Unless you have money to burn, I doubt you can buy 2 of the highest end V5 when they come out. Let's assume you get the cheapest V5s instead. I bet at that moment, a same-priced V3 will be faster because their prices drop. Let's assume you buy the basic V5 anyway. Many months later when you have the money to get the top of the line V5, won't the V6 be soon on the horizon anyway?

Only if the best V3 or V4 you can afford has too little performance for you and/or has very few faster models above it, you can get the mindset that you should wait.

It's my two cents, if you have money to blow and can recuperate the investment in less than a year, go to town


----------



## geort45

Quote:


> Originally Posted by *SRPlus*
> 
> I'm in two minds about upgrading though, driving me crazy, got the savings, itching to burn haha...
> 
> I think its better to upgrade at the start of a new platform than at the end of one, and the 2011-3 Z10PE-D8 WS and X99-E WS have had their last CPU refresh as far as Xeons are concerned... Xeon V5's will need a new socket.
> 
> Plus with pci-e 4, NV-link server platforms, and the Xeon Purley's on the very near horizon, I'd feel a bit gutted forking out on a new workstation, being stuck on a 2011-3 platform and not being able to upgrade to the latest and greatest in the future.
> 
> My current system is ok, it does the job I built it for well, so although I'm itching to upgrade to the current Xeons, and 4x Titan Pascal, to cut my render times down to a third, It looks like I will be playing the waiting game again......
> 
> I decided not to upgrade to the 2011-3, decided not to upgrade to 4x GTX Titan X's ( Maxwell ) because the 46% performance increase over my old Titans rendering in Iray was not worth the £4K price-tag, but the newer Titan X would definitely be worth the upgrade, as it would cut my renders to a third, and pay for themselves in a few months, but there is a very high probability that we will be seeing a fully loaded GP102 next year..
> 
> Anyone else stuck in the in the upgrade now or wait feedback loop?


A dual E5 2699V4 would be about double the rendering performance of what you have now, for example


----------



## SRPlus

Quote:


> Originally Posted by *geort45*
> 
> If you have the money to buy the newest Xeon motherboard and CPU when they come out, go for it. I just wonder what the prices will be, a basic V3 is very expensive, just imagine a V5.
> 
> This is a game you can't always be winning, you can have a year of "being at the top" at most, and only if you get the newest platform as it comes out. Unless you have money to burn, I doubt you can buy 2 of the highest end V5 when they come out. Let's assume you get the cheapest V5s instead. I bet at that moment, a same-priced V3 will be faster because their prices drop. Let's assume you buy the basic V5 anyway. Many months later when you have the money to get the top of the line V5, won't the V6 be soon on the horizon anyway?
> 
> Only if the best V3 or V4 you can afford has too little performance for you and/or has very few faster models above it, you can get the mindset that you should wait.
> 
> It's my two cents, if you have money to blow and can recuperate the investment in less than a year, go to town


I think its still better to be building a new workstation at the start of a new platform. It seems to be very 2 years a new LGA socket its introduced, but pcie refreshes seem to happen every 4 years. Next year will be the start of both new platforms so it seems like the right time to upgrade.

I'd imagine the price of the V5's especially top end to be cold sweat worthy, but whenever I build a new system I try to put the top end components in so that in 3 or 4 years time when its time to think about upgrading again, I still have a high end system as far as productivity goes.

My 4 year old dual E5-2687W's still have better rendering times ( slightly ) than the new top end overclocked I7 at the moment, so although they were expensive at the time, they've paid for themselves over time.

So if I put Dual E5-2687W v5's in they should be good for a few years. As long as my system can still slightly outperform a single socket i7 4 years down the line for CPU rendering, I consider then a good investment. And I' d hate to be stuck with pcie gen 3, and not get all the potential out of pcie gen4 if I build now on the 2011--3 platform.

Faster render times whether its CPU or GPU means more time for more clients, so although these systems cost a lot to build if you want the latest and greatest, they do earn their own keep over time..

Its hard waiting, but next years seems like the logical way to go..


----------



## geort45

Quote:


> Originally Posted by *SRPlus*
> 
> I think its still better to be building a new workstation at the start of a new platform. It seems to be very 2 years a new LGA socket its introduced, but pcie refreshes seem to happen every 4 years. Next year will be the start of both new platforms so it seems like the right time to upgrade.
> 
> I'd imagine the price of the V5's especially top end to be cold sweat worthy, but whenever I build a new system I try to put the top end components in so that in 3 or 4 years time when its time to think about upgrading again, I still have a high end system as far as productivity goes.
> 
> My 4 year old dual E5-2687W's still have better rendering times ( slightly ) than the new top end overclocked I7 at the moment, so although they were expensive at the time, they've paid for themselves over time.
> 
> So if I put Dual E5-2687W v5's in they should be good for a few years. As long as my system can still slightly outperform a single socket i7 4 years down the line for CPU rendering, I consider then a good investment. And I' d hate to be stuck with pcie gen 3, and not get all the potential out of pcie gen4 if I build now on the 2011--3 platform.
> 
> Faster render times whether its CPU or GPU means more time for more clients, so although these systems cost a lot to build if you want the latest and greatest, they do earn their own keep over time..
> 
> Its hard waiting, but next years seems like the logical way to go..


As I said, if you have the money to buy a V5 mobo with top of the line Xeons when they come out, go for it.


----------



## Jirkas

Can someone tell me, what I have to check, if I want to buy a second hand xeon 2670 v1. I don't understand the C1,C2 story. Do more editions of 2670 v1 exist? A german site says this:

SR0KX -> C2
SR0H8 -> C1 -> hier dürfte es einige Probleme bei USB 2.0/3.0 gegeben haben. Details sind mir aber nicht bekannt.
Wenn ihr also die Wahl habt -> C2

If it is possible, buy C2 !


----------



## geort45

Quote:


> Originally Posted by *Jirkas*
> 
> Can someone tell me, what I have to check, if I want to buy a second hand xeon 2670 v1. I don't understand the C1,C2 story. Do more editions of 2670 v1 exist? A german site says this:
> 
> SR0KX -> C2
> SR0H8 -> C1 -> hier dürfte es einige Probleme bei USB 2.0/3.0 gegeben haben. Details sind mir aber nicht bekannt.
> Wenn ihr also die Wahl habt -> C2
> 
> If it is possible, buy C2 !


There are "revisions" or "versions" for each CPU, just like when mobo manufacturers release rev 1.0 1.1 etc to address issues without changing the overall clock, cache etc, it's the same model but with "less bugs" so to speak. However they are FULL RELEASE CPUs, not ES, so it's not like a C1 CPU is a beta version... they all work but SOME TWEAKS AND FIXES can come later.

I personally have 2670 V1 C1 on the Z9

Download CPU-Z and check the value in "Revision". The CPU you buy should be the same Revision


----------



## Jirkas

Hi Guys,
what do you think about 2011-3 E5-2683V3 ?
Single E5-2683V3 - 17955 point
Dual E5-2683V3 - 22362 points

Price 600Eur Ebay

Performance
# of Cores 14
# of Threads 28
Processor Base Frequency 2 GHz
Max Turbo Frequency 3 GHz
Cache 35 MB SmartCache
Bus Speed 9.6 GT/s QPI
# of QPI Links 2
TDP 120 W
VID Voltage Range 0.65V-1.30V


----------



## Jirkas

It is interesting :

Single E5-2683V3 - 17955 point
Dual E5-2683V3 - 22362 points

Only 5000 points more







for price 600Eur Ebay







.


----------



## geort45

Quote:


> Originally Posted by *Jirkas*
> 
> It is interesting :
> 
> Single E5-2683V3 - 17955 point
> Dual E5-2683V3 - 22362 points
> 
> Only 5000 points more
> 
> 
> 
> 
> 
> 
> 
> for price 600Eur Ebay
> 
> 
> 
> 
> 
> 
> 
> .


5000 points more in what benchmark?


----------



## Jirkas

https://www.cpubenchmark.net/


----------



## geort45

Quote:


> Originally Posted by *Jirkas*
> 
> https://www.cpubenchmark.net/


Bleh. It al boils down to what are you gonna use them for. For 3d rendering it's almost a 2x increase going from 1 to 2 CPU's...


----------



## Jirkas

I don't know, you are maybe right. I am searching now, how much cores AfterEffects CS6 support...

2x 2670 250 EUR oder 1x 2683v3 600 Eur. Maybe it is better to buy 2011-v3, because you have less problems with the graphic card compatibility. Somewhere I have read, that Z9PE-D8 WS does not detect many graphic cards, if you use v1 processors ...

Or I can wait for better offer - I am hobby filmaker and it does not matter, if I have my new workstaion now or in 6, 12 months. Now I have i7 3770 ...


----------



## geort45

check offers a couple months and see if those prices are normal so you can detect a real offer

v3 is for Z10 motherboard though


----------



## Jirkas

DDR4 RAM is expensive, DDR3 is very cheap ...

Z9 is always better choice for the money.


----------



## Jirkas

And what do you think about Z9PE-D16? 16 RAM Slots ...


----------



## Wam7

Quote:


> Originally Posted by *Jirkas*
> 
> Can someone tell me, what I have to check, if I want to buy a second hand xeon 2670 v1. I don't understand the C1,C2 story. Do more editions of 2670 v1 exist? A german site says this:
> 
> SR0KX -> C2
> SR0H8 -> C1 -> hier dürfte es einige Probleme bei USB 2.0/3.0 gegeben haben. Details sind mir aber nicht bekannt.
> Wenn ihr also die Wahl habt -> C2
> 
> If it is possible, buy C2 !


C2 has VT-d (Virtualisation) which is broken in C1 steppings. VT-d is only useful if you are running Virtual Machines i.e. VMWare. I don't and never plan to so picked up a couple C1 2670's for $100 as they are cheaper than the C2.

I would also check that the application you are using is fully multi-threaded, as many are not. I'm not sure After Effect CS6 is so you could be wasting your money throwing lots of cores at it.


----------



## Farmfield

Hi guys,

I'm so confused right now, hehe, because I thought I had issues and now I wonder if I maybe don't have issues or if everyone here has issues. And no, I'm not talking about that kinda issues, though I have those too...









So this is how it is. I'm on a Z9PE-D16, but I ran a Z9PE-D8 WS over the weekend, with the exact same problem/non-problem...

Up to now I've run a i7-4790K, 32Gb DDR3 1600Mhz RAM, a GTX 970 and I had a Cinebench OpenGL score at 149 fps, a Passmark 2D score of 1065 and 3D score of 10047. Now, running dual E5-2670v2, 64Gb DDR3 ECC 1333Mhz RAM, a GTX 1070 and I have a Cinebench OpenGL score of 87, a passmark 2D score at 504 and 3D score at 8445...

EDIT:

Two hours with ASUS support on the phone today, troubleshooting, messing with BIOS settings, etc. - the tech was stumped, it should work, but it doesn't. Yeah, I'm dropping Asus for a Supermicro board, 20+ years messing with hardware, I never had so much issues with a mainboard - or any hardware, really... Broken, defect, of course, but not with a working board.

EDIT II:

Something is weird as F! because I find more and more people with these bad results using these Z9PE boards. Does anyone here get reasonable results in Cinebench and/or Passmark 2D/3D..?


----------



## Homerlovesbeer

I just ran Cinebench OpenGL and scored 85.88. Dual 2670's and an R290x.

Passmark 2D score 433
3D score 6716


----------



## Farmfield

You get 20 fps better than my 1070 in Cinebench (and I should decimate the R290X) but the R290X should still be around ~120 fps and your Passmark score should be double the score you got too.

I'm beginning to think this weirdness is about something else, probably the chipset and/or the PCIe controller, because I get wildly inconsistent scores in GFXBench and CompuBench, on most scores I'm on the baseline, on some I'm significantly better, on some significantly worse. And my GPU never maxes out, in Cinebench it's at 60% max at most, in Passmark 2D I have no GPU load whatsoever and under 50% load in the 3D bench...

So this is not about my mainboard - though I kinda knew that as I checked with a D8 WS too, exact same performance - and it's not about the GPU as I checked both a GTX 970 and GTX 1070 and both underperform badly in these tests.

I would really want to see the results from someone with a non Asus board, but I haven't managed to find anyone yet...


----------



## Farmfield

Now I got a reply from a guy on a Gigabyte dual 2011 board. Running a GTX 980, he gets 65 fps in Cinebench - and with that I can conclude, having seriously checked everything, talking to a lot of different users, this is something with the Intel C600 series chipset and/or the manufacturer BIOS settings. I know for sure it's not the GPU as I checked two high end GTX cards myself on two motherboards _ add to that everyone else's similar scores. It's also not connected to either 2600 or 2600v2 series, it's not connected to ram manufacturer, if it's ECC or non-ECC, neither is it connected to RAM speed.

I also tried the proverbial "everything" in BIOS settings, and the only thing I managed to get it worse benchmarks, hehe... So a tip is to keep the BIOS settings pretty vanilla, though disabling all the crap you don't use.

But let's be clear, in most cases, most benchmarks I've run, the GTX 1070 I run now performs in line with the median of tested 1070 cards - with some exceptions where I get either much higher or much lower score - but overall, it's pretty much where it should be in regard to performance, be it DirectX, OpenGL, OpenCL and CUDA.

So yeah, case friggin closed and I'm keeping this Z9PE-D16 board, switching to another manufacturer won't solve this (pretty clearly, small) issue. What I will do, though, is test with an additional 1070 card, I was planning to add one more anyway, and I'll report back with any findings.


----------



## Magos

Cinebench score (nothing overclocked...):



Passmark:

http://www.passmark.com/baselines/V8/display.php?id=68601435783

Nothing extraordinary but acceptable. Try to use High performance in power options if you haven't. Also what is your CPU (single core) score, because I believe the lower the number the bigger the bottleneck...


----------



## Homerlovesbeer

Try to use High performance in power options if you haven't.....

Where is this setting? In the bios?


----------



## Magos

I was referring to windows 7 options (control panel --> System and Security --> Power options --> select High performance).

Also in bios, if you like try:
- CPU Configuration: Server Class --> HPC
- CPU Power Management: Energy Performance --> Performance


----------



## halkx

Quote:


> Originally Posted by *Farmfield*
> 
> But let's be clear, in most cases, most benchmarks I've run, the GTX 1070 I run now performs in line with the median of tested 1070 cards - with some exceptions where I get either much higher or much lower score - but overall, it's pretty much where it should be in regard to performance, be it DirectX, OpenGL, OpenCL and CUDA.


same problem here with a gtx780 ghz ed and 2 2670s...3dmark06 below 17k and every time I run it it gets lower scores...same card in a s1151 with an i3 6320 gets over 24k...same board (s1151) with a 970 itx edition gets over 28k...I am waiting for a 980ti to arrive...I bought this second-hand at a very good price to replace the 780...did I make a terrible mistake??

did you do anyhting special for your 1070 to benchmark as it should on the z9pe?


----------



## Farmfield

Well, it's just really hard to know what the synthetic scores say in relation to everyday performance... As I said, I know - comparing times with other GTX1070s doing GPU rendering - that the GPU does do what I bought it for at the performance I was expecting - the question is how it performs shuffling polys around, and that's really tricky to test for too, at least for what I'm interested to know, viewport performance in Houdini, Nuke, Fusion, etc... At this point I've just let it go...

And I will update my CPUs and RAM soon (within a week or two), switching my 2670 v2's for 2667 v2's as well as switching my 1333Mhz ECC RAM to 1600Mhz sticks - it'll be interesting to compare the scores. I'll do a test switching the RAM, keeping the 2070 v2's, then checking both RAM set's with the two 2667 v2's... The thing is, it seems the GPU scores (at least the synthetic ones for OpenGL) do way better with higher single threaded performance and I've also seen numbers with GPU performance increasing with higher clocked RAM (thus my slower ECC might also be a culprit here) and there's just no reason for me not doing that too, maxing this box really is worth it for me, in regard to my daily work - so I'll quickly got that money back anyway...

Either way, I'll try to remember reporting back my findings to the thread.


----------



## Motion76

Hi! I'm new over here.

I'm a proud owner of a Z9PE-D8 since 2012. Now I'm getting into the GPU rendering world. And I wonder if there's any issue to have 4xGPUs and what is the maximum speed (x4,x8,x16) I can get. I was thinking on 1070 hybrid. What do you think? Thanks a lot!
https://pcpartpicker.com/b/MqnnTW


----------



## Farmfield

Quote:


> Originally Posted by *Motion76*
> 
> Hi! I'm new over here.
> 
> I'm a proud owner of a Z9PE-D8 since 2012. Now I'm getting into the GPU rendering world. And I wonder if there's any issue to have 4xGPUs and what is the maximum speed (x4,x8,x16) I can get. I was thinking on 1070 hybrid. What do you think? Thanks a lot!
> https://pcpartpicker.com/b/MqnnTW


I would go for CPUs with a bit better single core performance - doing high end graphics, I was surprised how much apps are still single (or badly-) threaded... After Effects and Media Encoder, as an example, are way slower after jumping to two Xeon's - though I had already more or less left Adobe anyway, and Nuke/Fusion are way, way, better threaded - Nuke especially is an absolute beast when let go with a lot of RAM and a lot of CPU cores, amazing performance increase... Houdini, which is my main application, is a mixed bag where running VEX or VOPs, the FLIP and Pyro solvers, took a huge performance jump - but it's also noticeable where many SOPs & DOPs are single (or badly-) threaded.

And you don't have to worry about the PCIe speed when doing GPU rendering as the i/o demands are just sooo much lower compared to gaming - which these ports/GPUs were meant to run. So that's a non issue. And I'm just about to buy a Redshift license, so I'm picking up a second Asus GTX 1070 Strix ROG today or tomorrow - they will give me some really nice rendering performance with RS - though sadly I'm out of PCIe ports, running 2 GPUs, as I run the Asus Z9PE-D16, I need 2 PCIe slots for sound and USB3 cards.

As for your build in general, personally I would go for Samsung SSD's and I would absolutely go for Samsung 950 Pro's (on m.2 adapters) for as many of them as I could - the performance is just so much better. I recently switched my main project and caching disk from a Samsung 850 Pro to a 950 pro on an Asus Hyper 4X adapter, and it's a noticeable difference when handling hefty data sets - lately, for me, pretty heavy alembic caches.

As a last note - Windows 7..? Why? Windows 10 is a noticeable more optimized OS, for those who stress over Cortana & that stuff (while not giving a ***** their Android/iOS devices spy even more on them) you can always use the DWS app ("Destroy Windows Spying") which handily disables all that crap. But ffs, let go of that crappy old Windows 7, there's just no reason running an outdated OS...


----------



## Fatsodonkey

Has someone tested any 1600 series cpu with this? Im thinking of buying this mobo and two 1600 cpu's.


----------



## Farmfield

ONE sixhundred (16xx) series as in ONE CPU and TWO sixhundred (26xx) series as in TWO CPUs.









And there are actually 4600 and 8600 series CPUs, if you wanna run more that two.


----------



## Fatsodonkey

16xx two cpu's.
Does this mobo even support 16xx? I know that intel c602 does support 16xx, but on asus site it shows only 26xx cpu's.


----------



## Farmfield

LOL, yeah, that won't work... Lets try it again. The ONE 6xx (16xx-) series cannot run other than as one single CPU, you need a TWO 6xx (26xx-) series CPUs to run them, on one board. If you want to run quadruple CPUs you need to use FOUR 6xx (46xx-) series CPU's, and so on.

But you can run one 16xx CPU on a dual LGA2011 board or a single 26xx CPU on a single LGA2011 board.


----------



## Motion76

Thanks Farmfied for your answer! I will try with 2 GPU and will increase the number next year if everything runs stable.


----------



## Farmfield

Yeah, at least there are no limitations on how many GPU's you can run, accept the number of PCIe slots you have. I just popped in a second GTX 1070 in my rig and now I'm trying to figure out how I can fit a third GPU on my Z9PE-D16 board... It gets kinda tricky when you - as I do - need a couple of PCIe slots for USB3, an m.2 adapter, etc... I have a couple of PCIe riser/extension cables on the way from China that'll hopefully allow me to work around the physical-space issue I have...


----------



## cgpyos

Hello, and sorry for my English in advance.
I'm about to build dual 2670 system based on Z9PE-D8
The question is, which cheapest RAM kits i can buy, to make whole thing work properly on this setup.
Also, it would be much appreciated, if someone could tell me in two words, what kind of problems i can face, when it will come to choose VGA.
Rig will be used for rendering.

Thanks!


----------



## Farmfield

Cheapest RAM is buying Registered ECC DDR3 RAM off Ebay. I payed just over €100 for 8x8Gb sticks, though I run a Z9PE-D16 so I bought two of those to get 128Gb.

Buying 8x16Gb RAM, to get 128Gb on the Z9PE-D8 is way more expensive even on Ebay, like €400 or so.


----------



## cgpyos

And considering VGA, does last NVIDIA series (10**) work as it should on this MB?


----------



## Farmfield

Sure. I run two 1070 and will put another 10xx card in this coming spring.


----------



## cgpyos

Quote:


> Originally Posted by *Farmfield*
> 
> Sure. I run two 1070 and will put another 10xx card in this coming spring.


I've inspected redshift forums a bit, seems to be good choice!

And what about non-ecc RAM? Anyone got good experience using it in dual Xeon rig with this MB?


----------



## Farmfield

I had 32Gb Corsair Vengeance DDR3 1600Mhz from my previous box, I sold that for €100 which is the cost of 64Gb 1600Mhz registered ECC - so even if you have a set of good RAM, it might be a better deal selling it and buying some dirt cheap registered ECC on Ebay.

If you don't and run the DDR3 you have, I doubt you'll ever notice any difference - Linus Tech Tips has tested RAm, done comparisons, time and time again they get the result that it barely makes any difference whatsoever. Here's one of the vid's...


----------



## ai1-fr

Quote:


> Originally Posted by *Zoner*
> 
> I personally couldn't NVMe to work with stock BIOS so I made a modified one, you can try it if you want. I've included the last version of AFUDOS that had the /GAN command, the website removed all version with that command.https://mega.nz/#!lYhT3Ibb!2flOAYfTfkzBDuvhUo8tfWlwfFKAU6OIDw3bSrDlkNQ
> ...


Hello Zoner,

I am a complete newbie to bios modification and all that. Right now I am tinkering with Toshiba XG3 NVMe (aka OCZ400) drive. I have updated the bios to the modified one, and installed W10 on the new drive and modified the CSM settings accordingly. Yet, it is impossible to boot from the NVMe. I am certainly missing something. Should I see the drive somehow in the Bios? Right now the drive is invisible in the bios, though Windows when booted from another drive sees it OK. Should I modify some PCIe settings or what?

Though the bios update process went fine with all OK messages, I wonder if, finally, I have the modified bios -- I do not see any indication of this. How could I verify that the bios I use indeed contains the NVMe block?

TThanks in advance,


----------



## GendoIkari

Has anyone else been having problems/issues with Windows 10 and the LSI Megaraid drivers?
I've been getting BSOD's and the drives attached to the LSI Megaraid controlled ports seem alot busier and slower than they used to be.
I've even swapped out my boot SSD's that are attached to the first two 6GB ports with no speed improvement.


----------



## GendoIkari

Also, for everyone on Windows 10 64bit, can you tell me what driver you are using for the onboard LSI controller?


----------



## scott1973

So just got this board and having a right nightmare with it. Can someone confirm if only using one CPU you have to have both 8 Pin CPU connectors attached as when I use just one it gets power to the board but just shorts out when you power on. If I unplug the CPU cable the machine will boot but not post. I only have one CPU 8 Pin lead hence why i'm asking.


----------



## Wam7

Quote:


> Originally Posted by *scott1973*
> 
> So just got this board and having a right nightmare with it. Can someone confirm if only using one CPU you have to have both 8 Pin CPU connectors attached as when I use just one it gets power to the board but just shorts out when you power on. If I unplug the CPU cable the machine will boot but not post. I only have one CPU 8 Pin lead hence why i'm asking.


I'm pretty sure you have to use both 8 pins connectors. I tested with one CPU and it worked fine with both connectors.


----------



## ViperJPB

Have seen a lot of posts related but none exactly answering my issues. Hoping to get a clearer picture: Samsung 960 Pro NVME Boot

- Just got a Samsung 960 Pro 1TB. Installed it with a Asus PCIe adapter. Getting full speed off it and all is great.
- Want to boot off it but even with the proper CSM settings the bios does not see it as a boot option.
- Cloned my boot drive to it and it is still not seen
- Have not tried installing windows directly to it just cloning with Macrium Reflect. However if bios cant see it I doubt that would change.

I'm on a version 1.0x board running Bios: 5503. 2xE5 2687W Rev C2. Dual 980Tis.

Do I need to update my bios and can I do so safely? I know people have had issues.

Any ideas or help on making the Samsubg 960 Pro visible as a boot option much appreciated. Thx.


----------



## nezbrun

Problem: I can't get the board to properly POST anymore, or get into setup, after a failed overclock. CMOS reset doesn't correct this.

Details...

I received this board a few days ago after making the mistake of purchasing a pair of E5-2696v2 QDUF ES processors on eBay a couple of months ago and failing to get either to work in a Supermicro X9DRi-LN4F+ (which does work fine with a pair of E5-2670v1's). This board was the chip vendor's recommendation.

To cut a very long story short, after lots of head scratching and random button pushing (wow, is this a flakey board or what?), I got it working reasonably reliably, loaded with both QDUF processors, 64GB of vendor certified 1600MHz DDR3 ECC RAM (8x8GB), MSI GTX-1070 Gaming X 8G and booting off a Samsung 950 Pro 256GB NVMe M.2 on a PCIe x4 NVMe adapater card. All PCIe cards installed on CPU 1 lanes. The GTX-1070 won't install in slot 1 as the back plate hits the DDR3 socket tabs, so I put it in slot 3, the next x16 slot. CPU cooling is air with Noctua NH-L12 which only just fit. PSU is Corsair HX1000i. BIOS version is 5802, this is the latest version as far as I can see, and it was shipped with the board.

Enclosure/case is NZXT S340 with a few mods to remove/add appropriate standoffs, but all my original setup was done outside of the case. Once I achieved a stable POST, the OS install and benchmarking and was done once in the case.

There is one caveat: I could only get it to fully POST and launch the OS (Windows 10 build 1607) with 11 of 12 cores on each CPU running. Bizarre, I know, but if I allowed all cores the machine would hang with Q-code DD. I could get it to boot consistently with 11 cores/socket. (I could run all 12 cores no problem with a single CPU on the board).

At this point I started benchmarking it, initially using CPU-Z as the tool of choice as it's quick to run and only tests basic CPU performance which is a reasonable indicator for initial sanity checks. I could consistently attain a CPU-Z score a tad over 28,500 with the 22 cores (44 threads) running. For reference, the 2 x E5-2670v1 comes in at around 20,000. In an AIDA64 stress test the processors sat at around 60 deg C and ran at a steady clock of 2800MHz. Cinebench CPU score was 2559.

These QDUF processors seem to turbo out at 3000MHz at standard 100MHz BCLK, unlike the kosher E5-2696v2 that max out at 3300MHz.

Then I went down the rocky road of over clocking. Reading of what others have achieved, I upped the BCLK to 105MHz, no problem, CPU-Z came in just touching 30,000 which I was pretty happy about.

Then it all went wrong, I upped the BCLK to 110MHz and since that point I've only managed to boot the system once, seemingly by fluke, and I managed to get into the BIOS setup and reset to defaults. After removing everything apart from one CPU and one RAM stick. It won't complete the POST, finishing with Q-code 32 for a few seconds, then rebooting itself and giving a consistent Q-code 00. By clearing the CMOS I can get it to attempt to go through a reasonably large number of POST steps, but it now always stops for a few seconds at Q-code 32, then restarts itself and sits at Q-code 00. I've tried taking the battery out and resetting the CMOS overnight, same problem. I put in a known working E5-2670v1, same behaviour.

So I think it's stuck in strange state, and I can't get into the setup anymore.

Anybody any suggestions?


----------



## nezbrun

Update to previous post, I have fixed it, after taking a week away from it. Sometimes that is the best way.

This time it was sleeves up and I took the board out of the enclosure, leaving a single CPU, one RAM stick, no PCIe cards. Same problem.

Tried to boot with no RAM sticks, of course it gives up right away.

Instead of putting the single RAM stick in the DIMM-A1 position, I chance sticking it in DIMM-B1 position, given I can't think what else I should be doing. It boots!

Very carefully I put everything else back in, one at a time, starting with all four RAM sticks on the first CPU.

I imagine that switching the RAM stick position somehow resets the machine enough to make it boot.

I haven't been able to boot it to the OS with both QDUF CPUs with all cores, I have had to set it to 11 cores/CPU. It hangs with Q-code DD if I enable all 12 cores with both CPUs. It'll boot fine with a single CPU with all 12 cores.

I've also overclocked BCLK to 105MHz, and thus far it seems fine and stable after a bit of AIDA64 sitting at 59 deg C max CPU temps.

*Update...*

While having another go on BCLK today, I managed to cause the same problem as in the previous thread, despite multiple CMOS resets.

It turns out that I also had to completely remove _both_ CPUs and power cycle with neither CPU installed to get it out of the Q-code 32 followed by 00 loop. (I also had all the RAM out and all PCIe cards removed). Frustratingly, the board would no longer fully boot with the on-board VGA, although the Q-codes looked reasonable. I had to put the GTX 1070 back in to see a BIOS setup screen.

I did manage to get it up to 107MHz BCLK, but it failed at 108MHz. I dropped the 1600MHz RAM down to 1333MHz while doing this test. I've settled for 105MHz for now, that passed an eight hour full AIDA64 stress test.

This is probably the worst board I've ever had to use, it's just so indeterministic. Considering its maturity I'd have hoped for better :-(


----------



## asidscript

Hello All,

I need your help / or advise,
I am soon going to be an owner of the z9pe-d8 WS
Ive all all the hardware i need but i cannot seem to get enough info on what case to put this thing into.

Prefrably i would like to get it into a Fractal Define R4 but i have not been able to find anyone who has done this.
and so the two cased that i am left to work with are the corsair obsidian 750d or the Fractal xl r2.

I need case that is going to fit under a desk, so anything as big as a cosmos 2 will not work.
Silence is a factor i would like to count in but fitting this under my desk gets priory.

I reach out to all the Z9pe-D8 WS owners for advise.
Can people who have this board in a 750d or an xl r2 (or if anyone has heard / done a fractal define r4)
Please can you educate me on the matter.

I would also prefer not to have to drill holes, i understand that there is an issue with the stand offs on all these cases.
Can you also elaborate on that too please.

Your feed back is very much appreciated.

Regards
Dan


----------



## Magos

@asidscript
Hi, it doesn't fit in a fractal R4 (for sure cause I have it with another system. It's clearly an ATX case) but it fits in XL R2. You could also use a nanoxia deep silence 5 (but it would be a little difficult with the cable management...) Also another good solution which I suggest, is Phanteks Enthoo Pro.

@nezbrun
Hi, It is written elsewhere in this thread that you have to reseat both cpus after overclocking failure in order to boot again properly.


----------



## Wam7

Quote:


> Originally Posted by *asidscript*
> 
> .... I would also prefer not to have to drill holes, i understand that there is an issue with the stand offs on all these cases.
> Can you also elaborate on that too please.
> 
> Your feed back is very much appreciated.
> 
> Regards
> Dan


You need to look for a case that says in it's specs that it supports SSI EEB form factor, like the Coolermaster Cosmos II. If it doesn't say so then you will have to drill some holes. I think I had to drill 3 holes in an old Coolermaster case I'm using for mine.


----------



## knopflerbruce

Is it possible to undervolt both CPUs with this mobo?


----------



## RushiMP

Hey guys, if anyone is interested I am parting out my workstation. I still have the motherboard with box and accessories, a pair of E5-2689 CPUs, and a pair of Supermicro SNK-P0050AP4 Heatsinks with Noctua fans. I have thrown it up on eBay but if you guys are interested I can share the links for the auction or just sell it here in the marketplace. My ebay handle is the same as my username here so, you can check it out if you like.

Not really selling it for the money (started the auctions at $1), just would like to see it put to good use.


----------



## knopflerbruce

Might be a good sale, I bid $200 and it didn't last long lol.


----------



## nezbrun

Quote:


> Originally Posted by *Magos*
> 
> @nezbrun
> Hi, It is written elsewhere in this thread that you have to reseat both cpus after overclocking failure in order to boot again properly.


Thanks, I guess my Googling chops just ain't good enough! As you've seen I figured it out for myself, that's a couple of days I'll never get back though :-(


----------



## franconia54

Hey guys, Im a happy new owner of a Asus Z9PE-D8 WS. Just updated to a workstation and had used an i7 system before.
My experience with such server oriented bios and setup is slightly, so I hope you can help me out a bit, setting up the bios and workstation correctly.
Im using it mainly for renderings with 3ds Max 2016 and Vray 3.40, CPU and GPU render engines.

My system is a
Board Asus Z9PE-D8 (Bios 5802)
CPU 2 Xeon E5 2670
RAM 8x8 GB RDIMM ECC 1600 Mhz from a HP Workstation
PSU Be Quiet 1000W
SSD 3x Samsung 850/830 Pro SSD
OS Windows 7 Pro x64
Cooler 2x NZXT Kraken X42

PCIE Slots
1. Slot Quadro K1200 (for Viewport)
2. Slot free
3. Slot GTX 1070 (for GPU rendering)
4. Slot free
5. Slot GTX 1070 (for GPU rendering)
6. Slot free
7. Slot Wifi Card

Questions 1:
In my actual setup the two 1070s are used with separate CPUs, since CPU1 is responsible for slot 1-4 and CPU2 for slot 5-7, right?
Is there any performance benefit in this constellation, or is it better to use both 1070s on one CPU (Slot 1 and 3, Quadro in slot 5). By the way SLI is not supported in 3ds Max.

Question 2:
After installing a fresh Windows 7 and installing the Nvidia drivers for the first 1070, everything was fine yet.
After installing the 2nd 1070 I had problems with very long boot times. I solved it by trying around and installing the sli bridge between the two cards, works perfectly.
But now with the 3rd GPU, the Quadro K1200 the problem is back and I have a really long booting time with a black screen. The system takes about 4-5 min instead of 60 seconds previously. I think it has something to do with my bios settings, cause there are almost default settings. Maybe someone has a similar system and can help me out with the bios settings. Thx a lot guys, your input is very welcome.


----------



## Aripug

Hi to everybody,
I'm building my secondary workstation and I'll use it as a little domestic renderfarm....I'll use a couple of E5-2670 on this nice motherboard....
I'd like to ask you an advice about the ram...
How muck and what kind of ram do you think is better to fit?
I'm looking the memory supported list http://dlcdnet.asus.com/pub/ASUS/server/ESC2000_G2/Manual&QVL/ESC2000G2_Z9PE-D8_WS_AVL_RDIMM_140829.pdf?_ga=1.19292797.432432923.1486213328 and I've just found a nice offer for this ram: M393B1G70QH0-CMA (the second of the list)
I need stability for this little workstation and before to decide I'd like to know what do you think about this ram...
Thanks again to everybody for the help and sorry about my poor english
Have a great day
Ari


----------



## Wam7

Quote:


> Originally Posted by *Aripug*
> 
> Hi to everybody,
> I'm building my secondary workstation and I'll use it as a little domestic renderfarm....I'll use a couple of E5-2670 on this nice motherboard....
> I'd like to ask you an advice about the ram...
> How muck and what kind of ram do you think is better to fit?
> I'm looking the memory supported list http://dlcdnet.asus.com/pub/ASUS/server/ESC2000_G2/Manual&QVL/ESC2000G2_Z9PE-D8_WS_AVL_RDIMM_140829.pdf?_ga=1.19292797.432432923.1486213328 and I've just found a nice offer for this ram: M393B1G70QH0-CMA (the second of the list)
> I need stability for this little workstation and before to decide I'd like to know what do you think about this ram...
> Thanks again to everybody for the help and sorry about my poor english
> Have a great day
> Ari


I'm running Corsair Vengeance 1600Mhz without problem, though I suspect the vast majority of memory out there would work stabily. I would just get the cheapest 1600Mhz I could find.


----------



## Aripug

Thanks a lot about your fast reply,
Doesn't matter if they are ECC or not?
Thanks again
Ari


----------



## Wam7

Quote:


> Originally Posted by *Aripug*
> 
> Thanks a lot about your fast reply,
> Doesn't matter if they are ECC or not?
> Thanks again
> Ari


The board will take either ECC or Non-ECC, just don't mix them together at the same time. I'm using Non-ECC.


----------



## dansv8

Hi Guys,

Have just setup my system and am wandering whats a normal CPU voltage for the E5 2670,
I have it all set to auto, and notice CPU #2 runs more voltage and subsequesntly more heat then #1
Is this normal ?
Should i just look to set voltages manually and keep them even ?

Thanks for the time

Dan


----------



## Aripug

Yes, thanks a lot








Now I've just to found a good eeb case









P.S
The new gpu like 1050, 1060, ecc ecc (the pascal series) will fit without issues?
Thanks
Ari


----------



## dansv8

ive got my board sitting in a thermaltake P5 case, it did require modifying to fit, however its a big open air case and works a treat so far.


----------



## nezbrun

Quote:


> Originally Posted by *franconia54*
> 
> Hey guys, Im a happy new owner of a Asus Z9PE-D8 WS. Just updated to a workstation and had used an i7 system before.
> My experience with such server oriented bios and setup is slightly, so I hope you can help me out a bit, setting up the bios and workstation correctly.
> Im using it mainly for renderings with 3ds Max 2016 and Vray 3.40, CPU and GPU render engines.
> 
> My system is a
> Board Asus Z9PE-D8 (Bios 5802)
> CPU 2 Xeon E5 2670
> RAM 8x8 GB RDIMM ECC 1600 Mhz from a HP Workstation
> PSU Be Quiet 1000W
> SSD 3x Samsung 850/830 Pro SSD
> OS Windows 7 Pro x64
> Cooler 2x NZXT Kraken X42
> 
> PCIE Slots
> 1. Slot Quadro K1200 (for Viewport)
> 2. Slot free
> 3. Slot GTX 1070 (for GPU rendering)
> 4. Slot free
> 5. Slot GTX 1070 (for GPU rendering)
> 6. Slot free
> 7. Slot Wifi Card
> 
> Questions 1:
> In my actual setup the two 1070s are used with separate CPUs, since CPU1 is responsible for slot 1-4 and CPU2 for slot 5-7, right?
> Is there any performance benefit in this constellation, or is it better to use both 1070s on one CPU (Slot 1 and 3, Quadro in slot 5). By the way SLI is not supported in 3ds Max.


On another forum, I did some tests recently that you might be interested in with two GTX 1070 in HB SLI (and also basic SLI) with the cards on separate CPUs and on the same CPUs. For a Unigine Heaven test, I found that I achieved about a 40% improvement having both cards on the same processor, and a further 4% improvement using an HB rather than a standard SLI bridge.

Code:



Code:


GPUs                      1xGPU    2xGPU     2xGPU     2xGPU     2xGPU     2xGPU
SLI                       N/A      Off       Std       HB        Off       On
PCIe x16 bus              N/A      Same CPU  Same CPU  Same CPU  Diff CPU  Diff CPU

Unigine Heaven  fps       41.7     41.1      73.4      76.0      41.2      52.5

(Edit: the Unigine Heaven benchmark was done at 2160p).

For batch rendering, I found a benchmark called Blenchmark v1.06 which uses Blender v2.78a. You have a fair bit of control over how and where it is run. Once I'd figured out how to run it, here are the figures for the render:

Code:



Code:


CPU 2 sockets, 22 cores.......65.96s
GPU single....................61.55s
GPU dual SLI OFF (same CPU)...33.24s
GPU dual SLI std (same CPU)...33.53s
GPU dual SLI HB (same CPU)....32.93s
GPU dual SLI std (diff CPU)...33.11s
GPU dual SLI HB (diff CPU)....33.55s

Machine is Z9PE-D8 WS with 2 x Xeon E5-2696v2 ES QDUF Bclocked to 105MHz, 2.94GHz turbo to 3.15GHz (restricted to 11 cores/socket), 64GB (8x8GB) DDR3 ECC RAM @ 1680MHz, 2 x MSI GTX 1070 Gaming, Corsair HX1000i PSU.


----------



## nissin

I've read through about 50 or so pages on this thread and dug around on some other forums and hardware review sites, and I'm still confused on a few things about the Z9PE-D8 board.

1. Is it still in production? I can only find it 2 or 3 places and it's well over $1200 in each case.

2. If it's not in production, is there a new / updated board? The Z9PE-D16 appears to be the same board, but with more ram slots, correct?

3. I'm building a workstation rather than a server (I need lots of cores to for Modo renderings- no gaming) and this seems like the board to go with using 2x 2670v1's, would you guys agree or recommend something else? I've built a lot of computers before, never a server though. I'm only running an ATI W4300 for graphics for SolidWorks so PCIe slots aren't critical. My priorities for this new machine are:
-Stability. I can't be 27 hours into a video render and have the thing crash
-Under $600 new / $200 used would be ideal
-Reasonable to setup. I've read horror stories about bricked mobos during BIOS updates, unsolvable hardware issues, RMA after RMA, Windows
constantly locking up, etc... are those occurrences more of the rule than exception? I know you often hear all the negative reviews but nothing
positive when the product is humming along.
-I'd love an ATX form if available. I hate huge cases.

Thank you for the help


----------



## GendoIkari

Quote:


> Originally Posted by *nissin*
> 
> I've read through about 50 or so pages on this thread and dug around on some other forums and hardware review sites, and I'm still confused on a few things about the Z9PE-D8 board.
> 
> 1. Is it still in production? I can only find it 2 or 3 places and it's well over $1200 in each case.
> 
> 2. If it's not in production, is there a new / updated board? The Z9PE-D16 appears to be the same board, but with more ram slots, correct?
> 
> 3. I'm building a workstation rather than a server (I need lots of cores to for Modo renderings- no gaming) and this seems like the board to go with using 2x 2670's, would you guys agree or recommend something else? I've built a lot of computers before, never a server though. I'm only running an ATI W4300 for graphics for SolidWorks so PCIe slots aren't critical. My priorities for this new machine are:
> -Stability. I can't be 27 hours into a video render and have the thing crash
> -Under $600 new / $200 used would be ideal
> -Reasonable to setup. I've read horror stories about bricked mobos during BIOS updates, unsolvable hardware issues, RMA after RMA, Windows
> constantly locking up, etc... are those occurrences more of the rule than exception? I know you often hear all the negative reviews but nothing
> positive when the product is humming along.
> -I'd love an ATX form if available. I hate huge cases.
> 
> Thank you for the help


DO NOT BUY THIS BOARD!
Lots of people have had issues with this board, and I think my 6gbps sata ports (1&2?) are possibly failing.
Z10PE-D8 WS is the next iteration of this board, but really, I would urge you to stay away from such high priced boards. If and when they fail, you will be out quite a bit of money.


----------



## Wam7

Quote:


> Originally Posted by *nissin*
> 
> I've read through about 50 or so pages on this thread and dug around on some other forums and hardware review sites, and I'm still confused on a few things about the Z9PE-D8 board.
> 
> 1. Is it still in production? I can only find it 2 or 3 places and it's well over $1200 in each case.
> 
> 2. If it's not in production, is there a new / updated board? The Z9PE-D16 appears to be the same board, but with more ram slots, correct?
> 
> 3. I'm building a workstation rather than a server (I need lots of cores to for Modo renderings- no gaming) and this seems like the board to go with using 2x 2670v1's, would you guys agree or recommend something else? I've built a lot of computers before, never a server though. I'm only running an ATI W4300 for graphics for SolidWorks so PCIe slots aren't critical. My priorities for this new machine are:
> -Stability. I can't be 27 hours into a video render and have the thing crash
> -Under $600 new / $200 used would be ideal
> -Reasonable to setup. I've read horror stories about bricked mobos during BIOS updates, unsolvable hardware issues, RMA after RMA, Windows
> constantly locking up, etc... are those occurrences more of the rule than exception? I know you often hear all the negative reviews but nothing
> positive when the product is humming along.
> -I'd love an ATX form if available. I hate huge cases.
> 
> Thank you for the help


1. No it is not still in production which is why they cost so much. They are like hens teeth and with how cheap 2670's were/are this only inflates the price of dual 2011 motherboards more.

2. There are no 2011 motherboards in production only 2011-3. The D16 basically just has more RAM slots

3. The 2670's are the bargain of the century so far. I bought 2 for less than $100 in total. My Z9PE-D8 cost me just over $200 and that was a bargain at that price but you'll probably be paying at least twice that I suspect. The only problem I had with my motherboard was when I tried to overclock with 105 Bclk and I had to remove both CPU's to get it booting again (I haven't seen any run 105 Bclk or higher) . Since then I've been running with a 104 Bclk and it has run faultlessly for months. I love it; added to that it is pretty much silent as only have an SSD, Passive GPU and silent PSU. The CPU's are cooled by Noctua D14's and the fans are permanently at 50% with my temperatures never go out of the 50's even max loaded.

There are a few other dual 2011 motherboards available though I suspect you will run into similar issues.


----------



## nissin

Quote:


> Originally Posted by *GendoIkari*
> 
> DO NOT BUY THIS BOARD!
> ...
> Z10PE-D8 WS is the next iteration of this board, but really, I would urge you to stay away from such high priced boards. If and when they fail, you will be out quite a bit of money.


I believe the Z10 is a 2011-3 socket, so that won't work for me.
Quote:


> Originally Posted by *Wam7*
> 
> ...The only problem I had with my motherboard was when I tried to overclock with 105 Bclk and I had to remove both CPU's to get it booting again (I haven't seen any run 105 Bclk or higher) . Since then I've been running with a 104 Bclk and it has run faultlessly for months.


What was your mult, or what was your resulting speed then?

From what I've read the oc for 2670v1's is minor, and seem to cause problems quickly. If I could oc to high 3's or low 4's I'd mess with it, but if I'm just going to get .1-.3 out of it, I'm going to leave it alone for stability and thermals.

I'm going to keep researching, but I think I'm leaning towards a new or refurb'd Z9PE-D16. I was thinking used, but having dealt with bent pins before I don't want to risk buying a used board with bad pins. Are there no dual cpu workstation boards that are as stable or easy to use as a z77 or x99 for example?

Thanks for the input guys


----------



## Houseofdreams

Hi guys..

I've been using this board for a couple of years now without any problems, but the time to upgrade has come.

The upgrade path goes beyond the scope of this topic, but I wanted to re-use this board in a server build. I only have one question:

The new server PSU has only one CPU power connector, but as I'm only using one processor at the moment, is it needed to have these two CPU power cables connected? Logic tells me it's only needed for each processor, and since I have only one installed... Or is this some kind of dumb logic?









Kind regards


----------



## GendoIkari

Quote:


> Originally Posted by *Houseofdreams*
> 
> Hi guys..
> 
> I've been using this board for a couple of years now without any problems, but the time to upgrade has come.
> 
> The upgrade path goes beyond the scope of this topic, but I wanted to re-use this board in a server build. I only have one question:
> 
> The new server PSU has only one CPU power connector, but as I'm only using one processor at the moment, is it needed to have these two CPU power cables connected? Logic tells me it's only needed for each processor, and since I have only one installed... Or is this some kind of dumb logic?
> 
> 
> 
> 
> 
> 
> 
> 
> 
> Kind regards


I actually ran with one CPU till I got the second one, I don't recall if I had to connect the second 8 pin for that thou. Even if it isn't needed, the mobo might be finicky and still want it connected.[


----------



## GendoIkari

Has anyone gotten a 1050 or higher to work in this board? I was thinking of swapping all my gpu's out for a 1050 or two.


----------



## Aripug

Hi to all of you!
Today I started to by the little renderfarm and I had a little issue during the assembling my z9pe on the enthoo pro, one hole of the board was "outline"....the case haven't the hole on the same line of the board, just one.....

Is it happened to you too?

Sorry about my poor english

Cheers









Do you think is it ok anyway to fix the board without that screw or it is better to modify the case? Thanks in advance for any advice


----------



## Aripug

Quote:


> Originally Posted by *GendoIkari*
> 
> Has anyone gotten a 1050 or higher to work in this board? I was thinking of swapping all my gpu's out for a 1050 or two.


I will use a gtx1050 2gb as viewport at the beginning......I'll let you know how it will work


----------



## nissin

@Aripug, I'd say you're fine as long as you put a thin plastic cap over the one stand off, or something similar so that you don't have the metal standoff rubbing on an electrical component on the back of the board and causing damage.

If you choose to totally remove the standoff rather than cap it or use a plastic piece, just be very careful when you're plugging in components in that area as the board is going to flex a lot more without that support. Or, maybe find a small plastic piece that's the same thickness as your standoff and use double-sided tape to fix it in place.


----------



## Aripug

Hi Nissin,
Thanks for the reply








To avoid any damage i removed the standoff that wasn't alligned with the hole....but it still remain the "flex problem"....I'll solve it in a way or another








THanks again


----------



## GendoIkari

Alright thank you. I was going to replace my six video cards with one or maybe two 1050 4GB ones.


----------



## DimitarBerbatov

Hi, my board is Z9PA-D8

1. I initially plugged a single module 16gb ddr3 1600mhz 1.35v Crucial in the correct dimm slot. There are no Crucial memories on the support list of Asus. However the board booted to Windows with no problem and worked fine. The next time I powered it on it just goes to B7 error and even if I change to a regular UDIMM I still have to clear the cmos to boot. Now this 16GB module doesn't go to post even with a cleared CMOS. This leaves me in a s**ty situation as I don't have a document for the purchase and no other compatible board to test the module. Did someone had a similar experience? Is there something I can do to use the module? I've reseated the CPUs, used a different case and PSU, tried with both onboard or external gpu...

2. I use 2x E5-2670 and I get a Cinebench R15 result of around 1800cb while on Asrock X79 board both CPUs can go over 1100 with boost to all cores. So is there an option for that in this board? Is there another way to get more performance or the same performance with less electricity as the CPUs have a headroom for it? The BIOS is somewhat overcrowded for a guy using only consumer boards till now.

3. Can I have a faster boot process on Win 10 without the initial checks and restarts?


----------



## cgpyos

Hey there, recentely i've decided to make some speed tests for my Samsung SSD 850 EVO, and figured out, that my results are pretty much worse than those, i see in reviews.

In bios SATA mode set to AHCI, running on win7.
SSD connected with 6GB/s SATA cord to Intel controller I believe (blue one).
Tried to switch connection to Marvell and Asmedia, got even worse results.
Here's screen of my benchmark below.

Any suggestions?


----------



## nezbrun

Quote:


> Originally Posted by *cgpyos*
> 
> Hey there, recentely i've decided to make some speed tests for my Samsung SSD 850 EVO, and figured out, that my results are pretty much worse than those, i see in reviews.
> 
> In bios SATA mode set to AHCI, running on win7.
> SSD connected with 6GB/s SATA cord to Intel controller I believe (blue one).
> Tried to switch connection to Marvell and Asmedia, got even worse results.
> Here's screen of my benchmark below.
> 
> Any suggestions?


Here is mine, 2TB Samsung 850 EVO.

[2 x E5 2696v2 ES QDUF OC'd BCLK 105MHz, 64GB RAM @ 1680MHz 8 x Hynix HMT31GR7EFR4C-PB, SSD in SATA6G_1 Blue connector on Intel RSTe (running in RAID mode for 4 x 5TB HDD in RAID10, but 2TB Samsung isn't running in RAID). GPU 2 x GTX 1070 HB SLi]


----------



## cgpyos

Thanks for feedback.
I've tried to put Intel RSTe drivers on my win 7, but had no luck. Some kind of platform incompatability error was appearing during installation.
Then I installed win 10 and things just got right. I'm getting advertised IOPS in benchmarks. Maybe issue was casued by win 7 sata driver called msahci.
It has been replaced by storahci in win 10.
Here's my results for the moment.


----------



## nezbrun

Something I discovered today after far too long was that this board seems to be far more stable booting (warm and cold) and coming in and out of S3 once switched to UEFI (I was using legacy).

To switch to UEFI I used one of the youtube videos, but made sure that my Win 10 install was running in safe mode when switching. To enable EFI I simply switched of CSM.

Of course, as well as being far more reliable to boot, it boots up a _lot_ faster too.

One further thing I have noticed is that in certain situations when overclocked with BCLK to 105MHz, I see tons of WHEA errors in the event viewer:

A corrected hardware error has occurred.

Reported by component: Processor Core
Error Source: Corrected Machine Check
Error Type: Memory Controller Error
Processor APIC ID: 32

The details view of this entry contains further information.


----------



## NapalmV5

since they removed all bioses from the support page.. whats the latest bios.. is it still the botched 5802 ?


----------



## sand74

All bios are on asus support page!!

Go to System - DOS









There are 17 bios files.


----------



## dansv8

Hi Guys,

Currently running win 10 with my Z9pe and duel 2670's
Had a couple people suggest maybe i should run windows server 2016.
System is mainly used for gaming and benching, but once the net gets upgraded i might start hosting a 7days to die server from it.

Just wandering if server 16 would actually yeild any performance benifits ?

cheers

Dan


----------



## NapalmV5

Quote:


> Originally Posted by *sand74*
> 
> All bios are on asus support page!!
> 
> Go to System - DOS
> 
> 
> 
> 
> 
> 
> 
> 
> 
> There are 17 bios files.


damn you had my hopes high for a 590x bios there.. so still 5802 oh well it is what it is

thanks much appreciated!

Quote:


> Originally Posted by *ViperJPB*
> 
> Have seen a lot of posts related but none exactly answering my issues. Hoping to get a clearer picture: Samsung 960 Pro NVME Boot
> 
> - Just got a Samsung 960 Pro 1TB. Installed it with a Asus PCIe adapter. Getting full speed off it and all is great.
> - Want to boot off it but even with the proper CSM settings the bios does not see it as a boot option.
> - Cloned my boot drive to it and it is still not seen
> - Have not tried installing windows directly to it just cloning with Macrium Reflect. However if bios cant see it I doubt that would change.
> 
> I'm on a version 1.0x board running Bios: 5503. 2xE5 2687W Rev C2. Dual 980Tis.
> 
> Do I need to update my bios and can I do so safely? I know people have had issues.
> 
> Any ideas or help on making the Samsubg 960 Pro visible as a boot option much appreciated. Thx.


950 pro is bootable in legacy mode going back to sandy bridge systems because it has its own nvme onboard whereas 960 pro isnt/doesnt

heres how they both perform on my z9:



edit: if you want to make 960pro bootable follow this guide:

http://www.win-raid.com/t871f16-Guide-How-to-get-full-NVMe-support-for-all-Systems-with-an-AMI-UEFI-BIOS.html


----------



## nezbrun

FWIW I originally used a 950 PRO to boot (legacy) mounted on a PCIE x4 card and it took forever to boot, it was very painful. I switched to an 850 EVO over SATA3, things were very, very much improved for booting in legacy.

However as I mentioned above in another post, I found this board to be incredibly unreliable at booting either from cold, warm or standby (S1 or S3) when in legacy that I nearly gave up on the board completely. Because of the nature of the boot failures, the Intel RAID 10 array kept rebuilding/verifying, and the volume shadow service kept losing its way so my incremental backups kept doing full file scans rather than VSS, wich is the difference between a few seconds and 10 hours in my system.

Once I'd switched to UEFI things have thankfully been much, much smoother (I have not tried UEFI with the 950 PRO, only the 850 EVO).


----------



## TheoChabawl

Hi everybody !

I just bought this motherboard. Everything worked fine until I add a new HDD "SEAGATE Barracuda 3To 7200RPM 6GB/s". Windows 10 pro doesn't boot when this HDD is pluged... And i can't access to the BIOS to check the boot priority. My Windows is installed on a SSD Crucial MX300 1To.

My system:

2x Xeon 2690
64g DDR3 ECC
GTX 1070 gaming X MSI
Windows 10 pro

My bios is up to date.

Any ideas ?

Thank you !!!


----------



## SRPlus

Hey,

Is anyone running Pascal cards in SLI on this motherboard?

Any problems with 2-way with a HB bridge over two CPU's ( slots 1 & 5 using 4 slot space HB bridge) , or 4-way with an EVGA HPC bridge?

Seriously considering replacing my aging 4x GTX Titan (2688 core ) GPU's for the new Titan Xp's ( 3840 cores ).

2 at first, another 2 later.

Mainly for GPU rendering iray, Vray-RT, but also for occasional gaming.

Cheers!


----------



## NapalmV5

Quote:


> Originally Posted by *SRPlus*
> 
> Hey,
> 
> Is anyone running Pascal cards in SLI on this motherboard?
> 
> Any problems with 2-way with a HB bridge over two CPU's ( slots 1 & 5 using 4 slot space HB bridge) , or 4-way with an EVGA HPC bridge?
> 
> Seriously considering replacing my aging 4x GTX Titan (2688 core ) GPU's for the new Titan Xp's ( 3840 cores ).
> 
> 2 at first, another 2 later.
> 
> Mainly for GPU rendering iray, Vray-RT, but also for occasional gaming.
> 
> Cheers!


i can assure you it works.. 1080ti sli does not work debug led went haywire and no windows boot

2way sli titan xp with 2x flexy/legacy sli bridges works as well.. i will receive the hb bridges tomorrow and it should work just as well


----------



## SRPlus

Quote:


> Originally Posted by *NapalmV5*
> 
> i can assure you it works.. 1080ti sli does not work debug led went haywire and no windows boot
> 
> 2way sli titan xp with 2x flexy/legacy sli bridges works as well.. i will receive the hb bridges tomorrow and it should work just as well


I currently have the 4-way EVGA High Pixel Clock SLI bridge.

So will you be running 2 of your TXP's in Sli using the HB bridge, and the other 2 just for GPGPU tasks? Does that work? Or is the 4-way sli screen grab from someone else's system?

I was planning either 2 setups..

One where cards in slots 1 & 3 are used in sli with a HB bridge, and cards in 5 & 7 are just for GPGPU

Second, where all cards are in 4-way sli on the older HPC bridge ( reduced to 2-way sli for gaming )

But I'm not sure the first option would work, driver issues etc..

Let me know how you get on with the HB bridge, and what, if any improvements it makes.

One other thing that still has me holding back on buying Titans again is NVidia's infamous strategy of releasing a cut down titan, then an equally fast ti version for a lot less $£$£, then a fully loaded titan at the same price, so will this be the last Pascal refresh, or will Nvidia pull out another Titan based on GP100 or with HBM2 memory in just a few months?

I know If I was one of the people that bought a Titan X Pascal 6 months ago, only for it to be discontinued and a better version for the same price, I'd be really peeved at Nvidia!

It happened to me when I bought the 2688 core titans, only for the Titan black to be lunched a few months later.

In the past I have ignored the "more money than sense attacks" about Titan owners as the majority of them use their cards for professional use and to save money on buying Quadro / tesla equivalents, maybe the 990ti is just as fast as the Titan Xm but that extra memory is what pro's want, and don't mind paying extra for.

This time around though, I think NVidia really put the middle finger up at the first buyers of Titan X Pascal, paying a lot more than 1080ti owners for just 1GB difference, then lunching a fully loaded Titan Xp 6 months later, and discontinuing the product, its a double insult.

Cheers!


----------



## NapalmV5

Quote:


> Originally Posted by *SRPlus*
> 
> I currently have the 4-way EVGA High Pixel Clock SLI bridge.
> 
> So will you be running 2 of your TXP's in Sli using the HB bridge, and the other 2 just for GPGPU tasks? Does that work? Or is the 4-way sli screen grab from someone else's system?
> 
> I was planning either 2 setups..
> 
> One where cards in slots 1 & 3 are used in sli with a HB bridge, and cards in 5 & 7 are just for GPGPU
> 
> Second, where all cards are in 4-way sli on the older HPC bridge ( reduced to 2-way sli for gaming )
> 
> But I'm not sure the first option would work, driver issues etc..
> 
> Let me know how you get on with the HB bridge, and what, if any improvements it makes.
> 
> One other thing that still has me holding back on buying Titans again is NVidia's infamous strategy of releasing a cut down titan, then an equally fast ti version for a lot less $£$£, then a fully loaded titan at the same price, so will this be the last Pascal refresh, or will Nvidia pull out another Titan based on GP100 or with HBM2 memory in just a few months?
> 
> I know If I was one of the people that bought a Titan X Pascal 6 months ago, only for it to be discontinued and a better version for the same price, I'd be really peeved at Nvidia!
> 
> It happened to me when I bought the 2688 core titans, only for the Titan black to be lunched a few months later.
> 
> In the past I have ignored the "more money than sense attacks" about Titan owners as the majority of them use their cards for professional use and to save money on buying Quadro / tesla equivalents, maybe the 990ti is just as fast as the Titan Xm but that extra memory is what pro's want, and don't mind paying extra for.
> 
> This time around though, I think NVidia really put the middle finger up at the first buyers of Titan X Pascal, paying a lot more than 1080ti owners for just 1GB difference, then lunching a fully loaded Titan Xp 6 months later, and discontinuing the product, its a double insult.
> 
> Cheers!


so after i checked tracking no hb bridge till friday so much for today

just business as usual nothing new except price.. in 2010 the cutdown fermi gtx480 was released in march full gtx580 released in november both at 500$.. a year later after your kepler cutdown titans were released they released the full titan black both at 1000$ then a year later titan x maxwell at same price.. the 1200$ cutdown titan x pascal was a year and half after so they were 6 months late with that and in the same time it took them to release gtx580 they released the full titan xp at 1200$.. yes theres that option that they could release a full 3840cc gp110/hbm2 to gamers/consumers if they wanted to or if competition forced them to but i dont believe thats going to happen.. this is probably the last pascal gpu.. then again they got something coming april 29th lol

the 4way sli is my screen shot and yes that would work.. i understand why youd want to sli slot 1 and 5 but i would suggest slot 1 and 3 which is cpu1 only better that way

you could quad sli at cpu1 x8 x8 x8 x8 and 2 or 3 gpus/raid card/nvme 950 pro at cpu2 but that would take a mining setup/pcie extensions/an extra psu or 2/etc

these titan xps will be split to 2x z170/6700k systems until x299

the numbers look great at aida64 gpgpu


----------



## SRPlus

Quote:


> Originally Posted by *NapalmV5*
> 
> so after i checked tracking no hb bridge till friday so much for today
> 
> just business as usual nothing new except price.. in 2010 the cutdown fermi gtx480 was released in march full gtx580 released in november both at 500$.. a year later after your kepler cutdown titans were released they released the full titan black both at 1000$ then a year later titan x maxwell at same price.. the 1200$ cutdown titan x pascal was a year and half after so they were 6 months late with that and in the same time it took them to release gtx580 they released the full titan xp at 1200$.. yes theres that option that they could release a full 3840cc gp110/hbm2 to gamers/consumers if they wanted to or if competition forced them to but i dont believe thats going to happen.. this is probably the last pascal gpu.. then again they got something coming april 29th lol
> 
> the 4way sli is my screen shot and yes that would work.. i understand why youd want to sli slot 1 and 5 but i would suggest slot 1 and 3 which is cpu1 only better that way
> 
> you could quad sli at cpu1 x8 x8 x8 x8 and 2 or 3 gpus/raid card/nvme 950 pro at cpu2 but that would take a mining setup/pcie extensions/an extra psu or 2/etc
> 
> these titan xps will be split to 2x z170/6700k systems until x299
> 
> the numbers look great at aida64 gpgpu


Yeah I know the history.. I had a pair of GTX 580 3GB before switching to the Titans. I followed all the Titan launches since, but didn't think the investment was worth it until now. E.g. the Titan X Maxwell was around 46% faster than my cards for GPU rendering with Iray, but it would cost 4K to replace my existing GPU's. Now one Titan Xp is twice as fast as one of my old Titans for GPGPU so its tempting to upgrade.

Titan launches have always been around the £800 mark from the right reseller in the UK or a little less, until Pascal hitting the 1.2K mark ( currently £1,159 here), so Nvidia launching the Ti with just 1GB less soon after, then a full-fat version just 6 month later would really put my back up the wrong way. Soo glad I didn't buy the first TXP.

As you say, only if AMD pull out a Titan killer, (which I seriously doubt ) will Nvidia respond with another faster GTX/Titan Pascal this year, or accelerate the Volta launch.

As you know the Z9PE-D8 has the two CPU's with 40 pci-e lanes each, CPU 1 controlling slots 1-4, CPU 2 controlling slots 5-7.

I already have 4x GTX Titans in 4-way SLI, and a dual PSU set up, 2x Antec HCP 1300 using the OC link to communicate load balance, and touch laminated wood the system has performed flawlessly for years, iray rendering using 4x GPUs+CPU's and gaming. Each card runs at 16x pci-e 3.0 so x16,x16,x16,x16 unlike the motherboard manual would suggest.

I know people have had a lot of problems with this MoBo and it being temperamental about which cards will run and in which slots, but luckily I've had non of that.

So I am hoping I can just swap my old Titans for 4x Titan XP's and use the 4-way HPC EVGA bridge I already have without issue. But it would be nice to take advantage of SLI in games, not just benchmarks. So was wondering if my option 1 would work&#8230;.

I know of a guy running 4x Titan Pascal ( old ) on a Z10PE-D8 WS running 4-way sli, again x16,x16,x16,x16 gen3, that actually got 4-way to work in some games using an Nvidia inspector hack.

I could test to see if I can use a 2-way bridge on GPU's 1&2 in SLI, GPU's 3&4 not used on my current set-up, but this system is my bread a butter, how I pay the bills each month, so very cautious about trying things that might brick the system..


----------



## NapalmV5

Quote:


> Originally Posted by *SRPlus*
> 
> Yeah I know the history.. I had a pair of GTX 580 3GB before switching to the Titans. I followed all the Titan launches since, but didn't think the investment was worth it until now. E.g. the Titan X Maxwell was around 46% faster than my cards for GPU rendering with Iray, but it would cost 4K to replace my existing GPU's. Now one Titan Xp is twice as fast as one of my old Titans for GPGPU so its tempting to upgrade.
> 
> Titan launches have always been around the £800 mark from the right reseller in the UK or a little less, until Pascal hitting the 1.2K mark ( currently £1,159 here), so Nvidia launching the Ti with just 1GB less soon after, then a full-fat version just 6 month later would really put my back up the wrong way. Soo glad I didn't buy the first TXP.
> 
> As you say, only if AMD pull out a Titan killer, (which I seriously doubt ) will Nvidia respond with another faster GTX/Titan Pascal this year, or accelerate the Volta launch.
> 
> As you know the Z9PE-D8 has the two CPU's with 40 pci-e lanes each, CPU 1 controlling slots 1-4, CPU 2 controlling slots 5-7.
> 
> I already have 4x GTX Titans in 4-way SLI, and a dual PSU set up, *2x Antec HCP 1300* using the OC link to communicate load balance, and touch laminated wood the system has performed flawlessly for years, iray rendering using 4x GPUs+CPU's and gaming. Each card runs at 16x pci-e 3.0 so x16,x16,x16,x16 unlike the motherboard manual would suggest.
> 
> I know people have had a lot of problems with this MoBo and it being temperamental about which cards will run and in which slots, but luckily I've had non of that.
> 
> *So I am hoping I can just swap my old Titans for 4x Titan XP's and use the 4-way HPC EVGA bridge I already have without issue. But it would be nice to take advantage of SLI in games, not just benchmarks. So was wondering if my option 1 would work&#8230;.*
> 
> *I know of a guy running 4x Titan Pascal ( old ) on a Z10PE-D8 WS running 4-way sli, again x16,x16,x16,x16 gen3, that actually got 4-way to work in some games using an Nvidia inspector hack.*
> 
> I could test to see if I can use a 2-way bridge on GPU's 1&2 in SLI, GPU's 3&4 not used on my current set-up, but this system is my bread a butter, how I pay the bills each month, so very cautious about trying things that might brick the system..


absolutely you can no different from my quad titan xms no different from your quad titans 4way sli works on these Xps with the included 4way bridge and i am sure it works better with the HPC EVGA bridge.. exactly the only issue is getting games to play nice past 2way sli.. since ive tested 1080ti sli if you have the means i wholeheartedly suggest going 7700k/z270/1080ti sli for pure silky fast gaming bliss.. 1080ti is castrated in every way possible but it dances around titan xp like a finely tuned car going thru corners as if they aint there and titan xp a constant voltage/clocks/loads roller coaster that has to be at least hybrid cooled and definitely needs hb bridge whereas 1080ti no need for hybrid cooling no need for hb bridge out of the box

something to be aware of.. pascal is basically a kepler 2.0 architecture.. maxwell (gm200) the 2 indicates how advanced the architecture and is the full realization of kepler (3072cc) the crown jewel what kepler couldnt be.. remains to be seen if volta will be to pascal what maxwell was to kepler if thats the case we could see a "gv200" volta at 4096cc/4096bit hbm2 on 16nm again they could release it if they wanted/if competition forced them to.. just in case you want to wait and see whats coming this summer/november/next march

those dual antec hcp 1300 nice! i got dual silverstone zm1350


----------



## SRPlus

Quote:


> Originally Posted by *NapalmV5*
> 
> absolutely you can no different from my quad titan xms no different from your quad titans 4way sli works on these Xps with the included 4way bridge and i am sure it works better with the HPC EVGA bridge.. exactly the only issue is getting games to play nice past 2way sli.. since ive tested 1080ti sli if you have the means i wholeheartedly suggest going 7700k/z270/1080ti sli for pure silky fast gaming bliss.. 1080ti is castrated in every way possible but it dances around titan xp like a finely tuned car going thru corners as if they aint there and titan xp a constant voltage/clocks/loads roller coaster that has to be at least hybrid cooled and definitely needs hb bridge whereas 1080ti no need for hybrid cooling no need for hb bridge out of the box
> 
> something to be aware of.. pascal is basically a kepler 2.0 architecture.. maxwell (gm200) the 2 indicates how advanced the architecture and is the full realization of kepler (3072cc) the crown jewel what kepler couldnt be.. remains to be seen if volta will be to pascal what maxwell was to kepler if thats the case we could see a "gv200" volta at 4096cc/4096bit hbm2 on 16nm again they could release it if they wanted/if competition forced them to.. just in case you want to wait and see whats coming this summer/november/next march
> 
> those dual antec hcp 1300 nice! i got dual silverstone zm1350


The reason I will stick with and most likely stay with a dual Xeon set up in the future is because the extra CPU cores make a huge difference rendering with V-ray/ mental ray. A 4 day animation render using a i7 will be cut down to 2 days using my aging 2687w Xeons, and the newer high end V4 Xeons are twice as fast as my old set up, so if I have a client wants the renders done in 2 days for a presentation, no problems, but a single socket i7 could mean delays.

If I upgrade to TXP's I will definitely water cool them. Unlike gaming where the load is distributed equally in SLI giving fairly low temps 50/60c, when rendering in Iray the GPUs are at a constant 100% load until the render completes. With 4x Titans that could be anything from 8 minutes to 30 minutes depending on the complexity/materials in scenes for a noise free render. So they get hot fast, and its already proven that the TXP stock cooler is not up to the job of keeping temps bellow to 70c mark at a constant 100% load, for longer life. Titans are proven to fail in render farms unless their temps are managed well, and kept low. I have had to ramp up the fans of the GPU's to 85% before render, and open windows sometimes in the summer, to keep the temps down, even though I have a huge case with industrial fans. So defiantly going all liquid the next build for a quieter life. If it were just for gaming I would not bother, but this systems priority is production rendering, gaming if I get the time..

I'm fairly confident now that I can just drop in 4x TXP's without issue, just would have been nice to take advantage of the HB bridge for gaming and leave the remaining 2 cards non SLI for GPGPU work.

I bought the Antec HCP PSU's because they communicate the load balance between the two PSU's. I have read and experience myself PSU failure just doing the wired hack to make 2 PSU's turn on at the same time, without the load balance communication running 4 high end GPU's, it eventually kills the PSU's.

Did the HB bridge make a difference to performance?


----------



## NapalmV5

Quote:


> Originally Posted by *SRPlus*
> 
> The reason I will stick with and most likely stay with a dual Xeon set up in the future is because the extra CPU cores make a huge difference rendering with V-ray/ mental ray. A 4 day animation render using a i7 will be cut down to 2 days using my aging 2687w Xeons, and the newer high end V4 Xeons are twice as fast as my old set up, so if I have a client wants the renders done in 2 days for a presentation, no problems, but a single socket i7 could mean delays.
> 
> If I upgrade to TXP's I will definitely water cool them. Unlike gaming where the load is distributed equally in SLI giving fairly low temps 50/60c, when rendering in Iray the GPUs are at a constant 100% load until the render completes. With 4x Titans that could be anything from 8 minutes to 30 minutes depending on the complexity/materials in scenes for a noise free render. So they get hot fast, and its already proven that the TXP stock cooler is not up to the job of keeping temps bellow to 70c mark at a constant 100% load, for longer life. Titans are proven to fail in render farms unless their temps are managed well, and kept low. I have had to ramp up the fans of the GPU's to 85% before render, and open windows sometimes in the summer, to keep the temps down, even though I have a huge case with industrial fans. So defiantly going all liquid the next build for a quieter life. If it were just for gaming I would not bother, but this systems priority is production rendering, gaming if I get the time..
> 
> I'm fairly confident now that I can just drop in 4x TXP's without issue, just would have been nice to take advantage of the HB bridge for gaming and leave the remaining 2 cards non SLI for GPGPU work.
> 
> I bought the Antec HCP PSU's because they communicate the load balance between the two PSU's. I have read and experience myself PSU failure just doing the wired hack to make 2 PSU's turn on at the same time, without the load balance communication running 4 high end GPU's, it eventually kills the PSU's.
> 
> Did the HB bridge make a difference to performance?


sorry for the late reply got overwhelmed.. sure do get the 2slot yes it does make a difference it depends on the game/settings/etc its more than just higher bandwidth its wired differently than 2x flexy legacy it forces the gpus to run at the same voltages.. at the moment i am enjoying witcher3 blood and wine at 4k/hdr/max settings/high quality drivers settings plus a bunch of mods easy breezy for titan xp sli @ 90-130fps


----------



## Aripug

HI!
I've this nice board from a while....it works perfectly excpet the usb ports...
I've a strange problem because sometimes, the usb port, won't recognize any external disk or usb memory stick...
Thanks for the help
Arian


----------



## keun

hi everyone!

my Z9PE won't post since yesterday... I changed a hard drive on my build, and when I wanted to restart it wouldnt post. no image nothing, I get the bc error... I cleared the cmos, removed all the gpus, only kep 2 sticks of ram... nothing!

please help! should I buy a new bios chip? or is this something else?

everything was running so smoothly when this happened... I don't get it


----------



## keun

anyone?

http://www.overclock.net/lists/display/view/id/6750013


----------



## murdaugh

I had that exact same problem last week and it was a bad ram stick on mine.

I dropped down to 1 stick of ram in A1 (even though I have 2 cpu's it worked with just 1 stick).

I then used the memory chart in the manual for adding the ram back 1 at a time until the boot got stuck at 6c again.

I was then able to add all 15 remaining sticks of ram and it booted fine every time as I added them. I just got the replacement stick in today and it is now back to a full 256gb.

If you still fail with 1 stick, try another stick, you might have the bad one in.

That is what worked for me with that error and mine also started after a power off. Would not come up again until I pulled the ram and found the bad one.


----------



## Wam7

Quote:


> Originally Posted by *keun*
> 
> hi everyone!
> 
> my Z9PE won't post since yesterday... I changed a hard drive on my build, and when I wanted to restart it wouldnt post. no image nothing, I get the bc error... I cleared the cmos, removed all the gpus, only kep 2 sticks of ram... nothing!
> 
> please help! should I buy a new bios chip? or is this something else?
> 
> everything was running so smoothly when this happened... I don't get it


Yeah the motherboard can be a bit temperamental. If I were you I would strip it down also removing both CPU's then rebuild. I know it's a pain but this is what I had to do when I a non-start issue due to overclocking.


----------



## tristansummers

SATA HDD on the SCU not mounting
I can't seem to figure out why SATA drives that show up in the SCU section of the BIOS are not mounting in Windows
They work in the other SATA ports so I guess it is a driver issue. They are not SAS.
Any ideas?
I'm not sure how to update drivers as the downloads have lots and nothing happens when you try to run them
Tris


----------



## ajbombadill

Quote:


> Originally Posted by *sand74*
> 
> All bios are on asus support page!!
> 
> Go to System - DOS
> 
> 
> 
> 
> 
> 
> 
> 
> 
> There are 17 bios files.


Sigh, yes I was hoping for a 590* update after reading your message, sigh.


----------



## linol

Someone running 2690v2 cpu's and have som stable overclock settings to share?


----------



## Wam7

Quote:


> Originally Posted by *linol*
> 
> Someone running 2690v2 cpu's and have som stable overclock settings to share?


With this board the most you will probably get is 104 Mhz on the BCLK. I have NEVER seen anybody run 105Mhz or higher. You might be the first but if your system totally locks up running 105Mhz or higher then you will probably need to strip it down, removing both CPU's to get it back up and running again. Mine has been at 104Mhz for over a year now.


----------



## ajbombadill

Thanks for the overclock info I will be trying this out soon, to see the performance I can achieve.


----------



## Wam7

Quote:


> Originally Posted by *ajbombadill*
> 
> Thanks for the overclock info I will be trying this out soon, to see the performance I can achieve.


Although there are myriads of settings in the bios there really is not much to change apart from the BCLK to 104.The only thing I would change is setting it to 'High Performance' which I think is in the power management settings. Also disabling Speed-step will give you better performance.

Depending on what you are doing then you might be better off disabling Hyperthreading; if what you do will max both CPU's to 100% ALL of the time i.e. rendering or batch processing some photo's then you can leave Hyperthreading ENABLED. If you are video encoding which does not CONTINUALLY max out both cores and keep usage constantly at 100% then you will get higher encode fps by disabling Hyperthreading.


----------



## linol

Thanks for the info


----------



## fickit1time

I know this has been asked but I am looking for a modded bios I can flash for samsung 960 nvme drives (apparently the 950's have onboard nvme but the 960's do not)

Can anyone please confirm they've been able to boot the 960's natively with a modded bios?
Much appreciated.

I have tried the method below but the bios still does not recognize the 960 nvme drive.
Quote:


> Originally Posted by *Zoner*
> 
> I personally couldn't NVMe to work with stock BIOS so I made a modified one, you can try it if you want. I've included the last version of AFUDOS that had the /GAN command, the website removed all version with that command.
> 
> https://mega.nz/#!lYhT3Ibb!2flOAYfTfkzBDuvhUo8tfWlwfFKAU6OIDw3bSrDlkNQ
> 
> Make a DOS bootable USB drive.
> Load files to USB and boot PC using it.
> On DOS prompt type "AFUDOS.EXE modified.rom /GAN" command. /GAN is undocumented key meant to flash all blocks regardless to any software locks. If the output is like
> 
> Reading flash ............... done
> Erasing All Block ............ done
> Updating All Block ........... done
> Verifying All Block .......... done
> 
> then your modified BIOS is flashed
> 
> Enter BIOS setup, on Boot tab enter CSM parameters it should look like this:
> Launch CSM [Always]
> Boot option filter [UEFI and Legacy]
> Launch PXE OpROM policy [Legacy Only]
> Launch Storage OpROM policy [UEFI Only]
> Launch Video OpROM policy [Legacy Only]
> 
> Other PCI device ROM priorty [UEFI OpROM]
> 
> Save and exit. Follow SSD manufacturer's instructions on installing windows.


----------



## fickit1time

I would like to go back to the standard bios from asus ver 5802

Anyone know if it is possible to reflash to 5802 cap bios file with afudos utility?
Can I rename the asus cap to rom (ie.5802.rom) and use afudos to update?

I tried using the flash util in the bios (Alt + F2) but it seems to flash the 5802 cap file too quickly (about 1 sec), I suspect it's not flashing it completely.

EDIT

For those who wish to downgrade or reinstall their bios I followed this 



 to remove the rom from the cap file using UefiTool
Keep the name of the rom file simple, e.g bios.rom

Then you need to create a dos usb startup key to boot (I used rufus)
You will also need to download afudos to do the update. (link)

Add both the rom file you extracted with UefiTool and the afudo.exe to your Dos bootable usb drive.

Restart your PC and press F8 to get to the boot menu, select the USB drive to boot the dos bootable drive.
Once dos has booted, type this command in.
afudos.exe bios.rom /GAN

The bios update should initialize and hopefully the results should go something like this.

Reading flash........... done
-ME Data Size Checking ... OK
Erasing All Block ...... done
Updating All Block ..... done
Verifying All block .....done

At this point the update has been completed sucessfully so restart your pc by holding the power button

Press Del to get into the bios (verify it is the bios you wanted) and set everything to default, save and restart


----------



## Homerlovesbeer

Quote:


> Originally Posted by *NapalmV5*
> 
> damn you had my hopes high for a 590x bios there.. so still 5802 oh well it is what it is
> 
> thanks much appreciated!
> 950 pro is bootable in legacy mode going back to sandy bridge systems because it has its own nvme onboard whereas 960 pro isnt/doesnt
> 
> heres how they both perform on my z9:
> 
> 
> 
> edit: if you want to make 960pro bootable follow this guide:
> 
> http://www.win-raid.com/t871f16-Guide-How-to-get-full-NVMe-support-for-all-Systems-with-an-AMI-UEFI-BIOS.html


Hi mate,

Your speeds are much better than mine with the same drive. What drivers and BIOS settings are you using?

I've never been quite happy with my speeds.


----------



## fickit1time

If you are using a 950, then those speeds you posted look pretty good.

I am getting a weird hd installation error.
I've set everything to default, except turned off secure boot and fast boot.

Whenever I install either windows 7 or windows 10, the installation process goes all the way through to selecting the drives and copying the extracted files from either usb or windows installation DVD.
Then it quits and reboots and I have to restart the whole installation process.

I've only gotten windows 7 to install and after installation, I noticed I have 1.2gb free of a 250gb SSD
After installing windirsat, there are 2 big files that take up more than 200gb of disk space, page file 100gb and hibernation file over 100gb
I've tried turning off hibernation in bios but then the installation fails again, only by turning it am I able to install windows.

I've also tried using another 500GB ssd and 1TB drive, each time windows 7 installs win 10 fails after installation it installs a 100gb file for both page file and hibernation.

I have 128gb of ecc memory installed and I can't seem to find any solution to the problem.
My system 2x e5 2687w, & 950 512gb, 960 1tb with pcie adapters, quadro k5000 video card, 1600w power supply, 2 noctua cpu coolers.

Any help appreciated.


----------



## tristansummers

I am going to try the hp thunderbolt 2 card from my z820 in my z9. It's a Intel 602 and it uses two usb2 slots. Only thing is if the driver works. Will let you know!!!
Secondly. Am i right in seeing that the only gen3 16x slot is the bottom one, which is to close to everything else r to put a double width card in? How can they say it is gen3 when all the slots are Gen 2??? I know for graphics use gen2 is fine but for compute it is a big deal. Or is the manual just wrong?


----------



## fickit1time

I believe all 7 ports are gen3, the blue are x16 and the black are x8, if you use 4 devices in the blue slots, they will all run at Gen3 x16 and if you run 2 device close together (1 blue & 1 black) then both slots will run at x8, its a little more complicated but look at the manual.


----------



## tristansummers

Er... If you read the manual, before telling me to read the manual, it has a block diagram that shows all slots as pcie2 not 3, hence my question.


----------



## fickit1time

Where does it say that it is PCIe 2 in the manual? Don't be an ******* when people are trying to help you out.

Specs for the z9pe-d8 ws
4 x PCIe 3.0/2.0 x16 (dual x16 or quad x8) *2
2 x PCIe 3.0/2.0 x16 *2
1 x PCIe 3.0/2.0 x16 (x8 mode) *

From the manual on page 2-18

2.5.4
The onboard PCIE 1 and 3 provide one x16 Gen3 link to CPU1 (Auto switch
to x8 link if PCIE 2 and 4 are occupied); The onboard PCIE 5 and 7 provide
one x16 Gen3 link to CPU2. These slots support VGA cards and various
server class high performance add-on cards.
2.5.5
The onboard PCIE 6 provides one x8 Gen3 link to CPU2; The onboard PCIE
2 and 4 provide one x8 Gen3 link to CPU1. These slot support VGA Cards
and various server class high performance add-on cards

PCIE 1
PCIE 2
PCIE 3
PCIE 4
PCIE 5
PCIE 6
PCIE 7

PCI Express x16 slot (x16 link)
PCI Express x16 slot (x8 link)
1 x PCI-E x 16 (Gen3 x16/ x8 link)
1 x PCI-E x 16 (Gen3 x8 link)
1 x PCI-E x 16 (Gen3 x16/ x8 link)
1 x PCI-E x 16 (Gen3 x8 link)
1 x PCI-E x 16 (Gen3 x16 link)
1 x PCI-E x 16 (Gen3 x8 link)
1 x PCI-E x 16 (Gen3 x16 link)


----------



## tristansummers

Apologies, I didn't mean to be rude. The block diagram, perhaps the most useful part of the whole manual, lists all as pcie2. Looking at the 2017 revision


----------



## tristansummers

That's why i am asking here, as this is the only place where people have verified everything. In assuming it is a mistake in the diagram.


----------



## fickit1time

You mentioned a Z9 in your post, do you have an Asus Z9PE-D8 WS motherboard?
The reason I ask is because there is no 2017 revision of that motherboard.

Here is the link to the manual, you should be able to use any of the ports on the motherboard, they are all PCIE3


----------



## tristansummers

Hi. Sorry I meant the 2017 version of the manual. If you look at the block diagram, in ask versions of the manual, everything but the bottom slot is described as pcie2. This seemed like a good place to ask for confirmation that other slots run at pcie3. Thank you for pointing me to the rest of the manual, that I have read. Sorry if i have seemed rude, i get that way when my intelligence is insulted. The manual is quite annoying as the hardware diagram is at the beginning but the block diagram is at the end. No point in the slots being pcie3 if the connection is pcie2. Just checking what you can and can't do with my 40 lanes from 2x2687s


----------



## tristansummers

So to start again!
Hi everyone, the block diagram in the manual actually says all the PCIe slots run at PCIe gen2, part from one at 8x on CPU2, and one at 16x which is the bottom slot.
Has anyone in the past four years checked this?
Or is it just the diagram is wrong?
The rest of the manual says they are all PCIe3 so I assume the diagram is incorrect but it would be good to check before I start puling my machine apart again.


----------



## Homerlovesbeer

Quote:


> Originally Posted by *tristansummers*
> 
> So to start again!
> Hi everyone, the block diagram in the manual actually says all the PCIe slots run at PCIe gen2, part from one at 8x on CPU2, and one at 16x which is the bottom slot.
> Has anyone in the past four years checked this?
> Or is it just the diagram is wrong?
> The rest of the manual says they are all PCIe3 so I assume the diagram is incorrect but it would be good to check before I start puling my machine apart again.




Page 2-18 of the owners manual.

http://dlcdnet.asus.com/pub/ASUS/mb/LGA2011/Z9PE-D8-WS/Manual/E12544_Z9PE-D8_WS_UM_V5_WEB.pdf?_ga=2.9911221.1670938178.1502785475-1045008395.1502785475

All Gen3 slots.


----------



## tristansummers

Not according to the block diagram at the end of the manual, the useful one that explains how the pcie lanes are actually shared out between usb, FireWire etc., hence asking if anyone had actually confirmed they are Gen 3, with actual data throughput tests


----------



## tristansummers

https://goo.gl/photos/LbRst6uM1KVvYAWy6


----------



## tristansummers

Slot 7 is the only gen3 x16 LINK


----------



## tristansummers

The slots might be gen3 but it's irrelevant of the link is Gen 2. You can't drive formula 1 on the m25


----------



## Homerlovesbeer

Quote:


> Originally Posted by *tristansummers*
> 
> The slots might be gen3 but it's irrelevant of the link is Gen 2. You can't drive formula 1 on the m25


I have used my Samsung 950 Pro Nvme drive in both slot 1, slot 3 and slot 7 with absolutely no difference in speed.

I've also used my R290x graphics card in slot 3 and 5 with the same result.


----------



## tristansummers

950 pro is 4 lanes at a speed that you wouldn't tell the difference between gen2 and gen3. Not proof really, but good to know.
It is probably pedantic, as most devices still don't take full adavantage of the benefits of PCIe 3 anyway. It is not just 8GB/s over 5GB/s but throughput and latency. My only real concern is for GPU compute. You should get direct links from GPU to Storage on the same Processors being actually used soon. For storage most devices are stil 4x and PCIe2 (eg Thunderbolt) anyway. You can only really put two double width GPUs in anyway, and even then with an IO card you hhave limited PCIe storage options, I'd certainly lose my 4 drive ioFX raid.

I just want to be able to fit two double width cards in for Octane and Resolve, but if the slots are going to be PCIe2 only there is no point buying newer cards in the future that can't utilise the extra bandwidth.

My RAID maxes out dead on 2000kb/s read with 4 x ioFX in the switchable slots (processor 18 lanes each, which is fine for 4 lane gen2 cards).
Be interesting to know if this hard limit is caused by the gen2 though, as I got 3000 read in my Z820, which was gen 3 throughout.
I know you can set them all to gen 3 in the BIOS, so they must be gen3, surely, it is just not what the block diagram says.

I presently have one Titan X and want to make sure it is in the fastest slot. In slot 5, where it is nowthe block diagram says that is 8 lanes at gen 3, but I can't fit it in slot 7. I'm concerned splitting up the ioFX raid across processors will slow it down a bit, and I was going to move my pagefile onto it....


----------



## Homerlovesbeer

Have you contacted ASUS and asked them to clarify?

I just upgraded from my 2 x 2670's to 2687w's today for a changeover price of $400 AU which isn't too bad.

Cinebench scores pretty impressive now.


----------



## tristansummers

contact Asus?
You have read some of this thread right?!
just asking for fellow nerd experience, rather than RTFM
IHRTFM!


----------



## Homerlovesbeer

I've got a copy of the manual that came with my motherboard at home. I'll see what that says.

Just because you aren't getting the answer you want straight away doesn't excuse your tantrum. Chill it down a bit mate.


----------



## tristansummers

Hi
Sorry if you are mistaking me. I am perfectly calm and intending to be friendly.
Can I ask why exactly you are going to look at your manual?
I am asking if anybody has tested to see if you get PCIe gen 3 speeds in the slots because the manual says they are only gen 2 in the block diagram
I am not asking for more quotes from the manual, but thank you all for taking the time to do so, albeit it somewhat wasted energy!
I have read the manual
I place myself before the mercy of those with greater experience. I have only just got my trusty, well sort of trusty, Z9 WS.
Here are many learned friends.
There was no tantrum, I am most chilled!
I have just had twenty years of people rather oddly avoiding the questions on forums!
much warmth and love guys, just checking a hunch.
Tris


----------



## Homerlovesbeer

And how do you expect us to test this?


----------



## wildpig

so what is the compatibility with windows 10 for this board (and the z9pe-d16)? I am seriously considering z9Pe-16 but I heard of acpi bios compatibility problem with win 10 so I didn't know if this has been resolved through bios update. if anyone has this board and running win 10 please reply. thanks


----------



## Wam7

Quote:


> Originally Posted by *wildpig*
> 
> so what is the compatibility with windows 10 for this board (and the z9pe-d16)? I am seriously considering z9Pe-16 but I heard of acpi bios compatibility problem with win 10 so I didn't know if this has been resolved through bios update. if anyone has this board and running win 10 please reply. thanks


I'm using Win10 Pro without problem for a over a year now, though it is version 1511 and I've disabled updates as everything works fine and this machine is pretty much only used for video work. I know of people having problems with subsequent updates with some older GPU's etc so your experience might differ depending also on the other hardware you have.

The only other hardware I have in this machine is a passive Nvidia GT720 GPU and a 256 Crucial M4 SSD drive.


----------



## fickit1time

I installed win 10 x64 fine with k5000 video card, didn't have any issues but I went back to windows 7 as I hate the constant updates and apps being installed without my control.

Wanted to let everyone know that it is possible to install samsung 960 pro 1TB nvme with a pcie adapter as the boot OS.
I used the clover method which requires keeping a small usb drive with the clover files plugged into the board at all times, there are 2 usb's on the main board itself unused.

I think the clover method is easier than modifying the bios which had some issues after inserting the nvme module that I didn't like.
I strongly suggest you save your current bios if you are going to use a modified bios.


----------



## Wam7

Quote:


> Originally Posted by *fickit1time*
> 
> I installed win 10 x64 fine with k5000 video card, didn't have any issues but I went back to windows 7 as I hate the constant updates and apps being installed without my control.


Just use something like W10Privacy https://www.winprivacy.de/english-home/ you can also disable all the Win10 telemetry stuff (aka Spyware)


----------



## fickit1time

Thanks! I'll definitely use that program when I install win10 on client pc's.
For me I still think Win7 kicks ass


----------



## Wam7

My girlfriend's PC is still using Win 7 and my work laptop is running it also as I had issues with some phone/tablet firmware updating programs on Win 10.

Up till recently I was still advising some clients to use Win 7 but the latest bout of ransomware has led me install Win 10 on most machines now.

Thanks for the info on the clover method as it may come in handy down the line as I will probably have the Z9PE for a few years yet.


----------



## wildpig

what bios ver is everyone running win10 on? the latest bios?

I have dual e5-2696 v2. intel s2600cp doesn't work with OEM cpu that's not on the "official" list, which the 2696 v2 is not even though it's not a QS or ES cpu. tried supermicro MB but they have acpi bios compatibility problem with win10 as well as having only s1 sleep and not s3. asrock MB also have issues with win10. s2600cp is stable with win10 with s3 sleep but intel is very strict with what cpu microcode they support. Hoping the z9pe-d16 would be good in having compatibility with win10 and s3 sleep


----------



## DimitarBerbatov

Quote:


> Originally Posted by *DimitarBerbatov*
> 
> Hi, my board is Z9PA-D8
> 
> 1. I initially plugged a single module 16gb ddr3 1600mhz 1.35v Crucial in the correct dimm slot. There are no Crucial memories on the support list of Asus. However the board booted to Windows with no problem and worked fine. The next time I powered it on it just goes to B7 error and even if I change to a regular UDIMM I still have to clear the cmos to boot. Now this 16GB module doesn't go to post even with a cleared CMOS. This leaves me in a s**ty situation as I don't have a document for the purchase and no other compatible board to test the module. Did someone had a similar experience? Is there something I can do to use the module? I've reseated the CPUs, used a different case and PSU, tried with both onboard or external gpu...
> 
> 2. I use 2x E5-2670 and I get a Cinebench R15 result of around 1800cb while on Asrock X79 board both CPUs can go over 1100 with boost to all cores. So is there an option for that in this board? Is there another way to get more performance or the same performance with less electricity as the CPUs have a headroom for it? The BIOS is somewhat overcrowded for a guy using only consumer boards till now.
> 
> 3. Can I have a faster boot process on Win 10 without the initial checks and restarts?


Anyone?


----------



## fickit1time

I was using 8x 16GB ECC (kingston kvr16r11d4/16 1.5v) memory fine but after a few years slot 8 stopped working for me, do you have them starting from slot A1 then slot E1 for 2, so on?
I would also put the basic timings in the the memory section of the bios and make sure its set to the 1600mhz speed.

I don't have the board anymore so can't remember where in the bios that is..
AI tweaker > Dram timing control, you can also adjust the memory voltage here


----------



## wildpig

got my z9pe-d16 mb. worked great with win 10 1703 .. using with with dual e5-2696 v2 cpu.


----------



## SRPlus

Just spotted this beauty this morning!

https://www.asus.com/Commercial-Servers-Workstations/WS-C621E-SAGE/

I'm drooling!!!!









Planning either WS C621E SAGE or Supermicro X11DPG-QT motherboard next year. But obviously the C621E has the advantage of SLI if I ever get time for gaming... plus a pair of Xeon Gold 6154's


----------



## xavierFR

Hi,

I have ASUS Z9PE-D8 WS and it is not BOOT.

The CPU fan and led turn 1 second max, and nothing more. no Bip sound ...
From i understand, it comes from PSU.

I have make few tests with ...

-PSU:
EVGA SuperNOVA 1600 T2, 80+ TITANIUM 1600W
Seasonic M12 II SS 750 M

-Memory in dual (not Mix):
DATARAM 8GB PC3 14900 E 11-12-E2
DATARAM 8GB PC3 12800R 11-11-E2
Kingston 8gb pc3 12800E 11-12-E3
Corsair revenge MZ16GX3M2A1866C9 (2x 8 Gb)

-GPU:
EVGA GTX TITAN
EVGA GTX TITAN SC
MSI GTX TITAN
Saphire HD 7850

-CPU
2 dual E5-2687W V1

In any way, it's not Boot.

I will buy Corsair CORSAIR AX1500i. because some people run with corsair ax1200i and i need power ...

Do you have any comments about PSU AX1500i or other high PSU power model?

Are there PSU list with ASUS Z9PE-D8 WS and recent for buy run for sure with ASUS Z9PE-D8 WS ?
What council about my DDR3 memory?

Thanks
Best Regards
Xavier


----------



## fickit1time

It does sound like your power supply, lets hope that is the issue and not the motherboard.
You have a mix of memory though, was the system running fine before?
Try with only 2 memory stick (Corsair revenge MZ16GX3M2A1866C9 (2x 8 Gb) in slots 1 & 2 and maybe 1 video card to see if the system does turn on.

You don't have to buy Corsair, as long as you buy a power supply with the right motherboard connections and enough power to supply your system components. I would buy something in the 1500W range since you are running 3 titans.


----------



## xavierFR

Hi,

Thanks for answer

I have buy the card to other man. a professional with server computer, and he have sale the motherboard of the server. the man will write me for confirm the Motherboard was running but with other PSU. I wait to know the PSU model.

I not have mix the memory model. but i have use different memory. i have use only 1 memory model by 2 memory stick or single... after, I have use second memory model ... but not mix memory model.

With or without video, nothing change.
but i keep munimum 1 CPU.
The led and fan run 1 second max...

for the moment i try to turn ON.
In Futur..., soon, i hope, My project is 4 GTX Titan. by 1 or 2 PSU.
I was thinking to use only 1 PSU , the EVGA SuperNOVA 1600 T2, 80+ TITANIUM 1600W.
If it need, i will use 2 PSU, Seasonic M12 II SS 750 M for Mother Board and CPU, and the EVGA SuperNOVA 1600 T2, 80+ TITANIUM 1600W for 4 EVGA Titan.
but if Seasonic and EGVA not turn ON the mother board, i hope to use Corsair CORSAIR AX1500i . but, i think, it will be not enough for 4 Titan...i think it will need EVGA SuperNOVA 1600 T2 for only Titan's. But it is the futur...

I continu to try to understand and to test...

Thanks


----------



## fickit1time

Have you tried resetting the bios (clrtc 1 pins), remember to remove the power cable.
Remove everything except 1 stick of memory in slot 1, 1 cpu and 1 video card and test the system.

Guess you will have to wait until you can try another power supply.


----------



## xavierFR

Hi,

Thank for answer

If i not make a mistake, the motherboard need to be with all the different voltages from PSU. The reset are only available with only boot problem from Bios. But, my first problem come from (as like) PSU, because, i not have all the voltage needed by the motherboard.

Clear RTC RAM (CLRTC1) process :
This jumper allows you to clear the Real Time Clock (RTC) RAM in CMOS.You can clear the CMOS memory of date, time, and system setup parametersby erasing the CMOS RTC RAM data. The onboard button cell batterypowers the RAM data in CMOS, which include system setup information suchas system passwords.To erase the RTC RAM:
1.Turn OFF the computer and unplug the power cord.
2.Move the jumper cap from pins 1-2 (default) to pins 2-3. Keep the capon pins 2-3 for about 5-10 seconds, then move the cap back to pins 1-2.
3.Plug the power cord and turn ON the computer.
4.Hold down the Del key during the boot process and enter BIOS setupto re-enter data.

Possible i can clear. But, the next operation will be not possible because i not have all PSU voltage and not long time 1 seconde. It will be impossible to the mother board to recovery Bios without all the voltage. Clear RTC RAM are for clear Bios parameter. But with out voltage, the motherboard can not make status, can not read Bios, and not read parameters. In my step with out voltage not long time, i not need Clear RTC RAM for the moment. maybe after, only if the next PSU run
True or not ?

I need 750W minimum

What PSU everybody use for Asus z9pe-d8 ws ?

Thanks


----------



## fickit1time

You can try a quick test to see if your power supply is working, it is not the best test but you can try.
Remove all the power connections from the motherboard.
On the 24pin connector, use a jumper wire or paperclip and put it in the green wire pin & black wire pin. 
Some power supply fans will only work when they reach certain temperatures so it's really not the best test anymore and even if the fan starts there could still be an issue with the power supply sending all the right voltages.

If you plan on using 4 titans you need a good power supply, at least 1500w or better.
Make sure the power supply has 2x 8pin cpu connectors and 8x pci-e (6 + 8pin) for your video cards.
I used a Lepa 1600W power supply

When you clear bios (using jumpers on board) the motherboard will reset the bios back to factory settings.
If your power supply is faulty it will not effect bios, the system will either startup or not.
Remember to try with 1 cpu, 1 memory and 1 video card.


----------



## xavierFR

Hi,
Thanks for answer.
i have seen few review of PSU.
The EGVA PSU 1600 are the most titanium power full i was find. it make arround 1620W.
i just have buy Corsaire Ax1500i for the motherboard. but the test show arround 1460 W. this PSU will come wednesday.

for the PSU with jumper wire or paperclip. it is a solution for second PSU for only GTX Titan, not use for motherboard.
The motherboard and PSU need to use this contact for communication with « PS_ON » and « PWR_OK » contact. by this way, jumper wire or paperclip are not possible for turn ON the motherboard. with out communication, the PSU will turn OFF.

I have test with only 1 Cpu... same problem.

you means Lepa 1600W power supply for only GTX Titan ?
no Lepa 1600W power supply for turn ON ASUS Z9PE-D8 WS motherboard ?

I need 750W minimum

What PSU everybody use for Asus z9pe-d8 ws ?

Thanks


----------



## fickit1time

You only need 1 psu for the whole system.
Don't need 2 psu...

Test your psu, disconnect all cables from motherboard,hdd, video card and use pin on black & green to see if your psu turns on. (



)


----------



## xavierFR

Hi,

thanks for answer
the PSU and all ot her component run. i have 2 computers, the second with Asus P9X79 E WS, and i use in this other computer for check (before and after Asus Z9PE-DE WS) .

Thanks


----------



## xavierFR

Hi,

Good new. the PSU was permit to turn ON the motherboard... Thanks for your supports









now, i try, but, i can not boot by USB for install Win 10 on Samsung SSD pro sata... i begin to update Bios, i will try UEFI...and after, to boot for install Win 10 on SSD... very not easy motherboard...

Thanks


----------



## fickit1time

You should be able to install windows 10 but I think you need to install the hard drive drivers if you use raid C600 or raid marvel drivers.
Download drivers from asus website.
I don't have the system anymore so I can't remember.

If you are installing uefi mode then you need to format the drive in uefi as well.
Use Rufus, in partition scheme use "mbr with uefi" and also at the bottom you can select "create bootable disk using iso image > select win10 iso "


----------



## xavierFR

Thanks, i will find information about it...

I was thinking to install 2 OS, Win7 and Win10. in first Win 10. but from i understand with you explain,maybe, to begin by Win7 by CD and after Win10 will be more easy ?


----------



## xavierFR

Hi,

does i make a mistake?

Asus z9-pe de8 ws have Smbios Legacy 2.3.
it is the minimum for Win10
http://dellwindowsreinstallationguide.com/uefi/

If it is 2.7 or greater you should have a UEFI BIOS with Secure Boot.
If it is 2.6 you might have an Early UEFI System (UEFI without Secure Boot) or only a Legacy BIOS. You will need to check your BIOS Setup.
If it is 2.5 you will have a Legacy BIOS.
If it is 2.4 you will have a Legacy BIOS and the processor should be checked for 64 Bit compatibility.
If it has an SMBIOS of 2.3 or less its below minimum system requirements for Windows 10.

2.6 version will be to much, Sandy cpu are not recognise.

i will like to put win10 with UEFI.
Does some one needed to upgrade our Smbios Legacy 2.3 to... 2.4 , 2.5 .. ?

Thanks


----------



## fickit1time

In my opinion windows 10 is terrible. Use windows 7 its still the best.
If you want to update the bios, the latest bios is 5802
I do not think don't need uefi boot though, you can still install windows 10 single or dual boot.
I don't remember but I might have turned off secure boot in bios.


----------



## xavierFR

Hi,

Thanks.
ok, i will not make to much complicate...
the Bios 5508 run.
I have find for Boot in UEFI and Dual... now, i need to test...
i wanted to keep win10 for everyday and Win7 for professional software and some other.

Win10 are terrible for not permit to obtain full performance. The true full performance with 4 SLI are no good. and some sofware lose some performance with Win10.

It is the why, i wanted Win 7 for full 4 way SLI performance for some video render software... and full performance from some software ...

Thanks


----------



## xavierFR

Hi,

for the people interrest to Install Win 10, in UEFI, GPT partition structures and boot by USB for installation...
With Bios 5508 version, Bios configuration need (I have use) ...

Launch CSM --> Always
Boot option filter --> UEFI and Legacy
Launch PXE OpROM policy --> UEFI only
Launch Storage OpROM policy --> UEFI only
Launch Video OpROM Policy --> Legacy only
Other PCI device ROM priority --> UEFI OpROM

Exit from Bios, and select (in Bios or after F8) to boot by the UEFI usb key bootable windows 10 ...

UEFIBootinstallWin10.JPG 182k .JPG file


----------



## Cogent

Hi folks!

I have few questions regarding Z9PE-D8 WS. I recently purchased this MBO and pair of E5-2670's. I have pair of GTX690's from my old rig.
Since I have boot problem with GTX 690 (only 2 succesfull attempts out of 30) I would like to know what is solution to this problem.

BIOS is lastest 5802, so that is not an issue. I see that ASUS offers vBIOS for GTX 690 which forces UEFI boot. Is this same solution as flashing PLX firmware or those two solutions are unrelated?

Thank you in advance for your responses.


----------



## Wam7

Quote:


> Originally Posted by *Cogent*
> 
> Hi folks!
> 
> I have few questions regarding Z9PE-D8 WS. I recently purchased this MBO and pair of E5-2670's. I have pair of GTX690's from my old rig.
> Since I have boot problem with GTX 690 (only 2 succesfull attempts out of 30) I would like to know what is solution to this problem.
> 
> BIOS is lastest 5802, so that is not an issue. I see that ASUS offers vBIOS for GTX 690 which forces UEFI boot. Is this same solution as flashing PLX firmware or those two solutions are unrelated?
> 
> Thank you in advance for your responses.


Can you boot with only one GTX 690?


----------



## Cogent

Currentlly, I can't. I tried many times after first two succesfull attempts, but now it only boots with onboard GPU. I trows B2 error every time with GTX 690.


----------



## Wam7

Quote:


> Originally Posted by *Cogent*
> 
> Currentlly, I can't. I tried many times after first two succesfull attempts, but now it only boots with onboard GPU. I trows B2 error every time with GTX 690.


The Asus Z9PE-D8 does not have onboard VGA (well mine doesn't), so check you are referring to this motherboard.


----------



## xavierFR

Hi,

I had 2 same problems but i not remember about code error...
(sorry for my english... I am French)

First problem.
From my experience, I build 4 way SLI GTX Titan Kepler. But with Win 10, or Win 2012 rt, and last 5802 Bios, I had a boot problem. 2 cards were not recognized and I had problem with win10 and Win 2012 rt.
I have use 5701 Bios. Now, I have no boot problem with Win 10 or Win2012 rt.

GTX 690 is a dual GPU, from Bios and Windows, it is as like 2 way SLI. 2 x GTX 690 is as like 4 way SLI GTX 690, I council you to put 1 card in Slot 1 , and second card in Slot 5.

Second problem.
It can be Pcie 2.0 or 3.0 version for yours GTX 690.
If you have Pcie 2.0 GPU card and use Pcie 3.0 slot, you will have a more bigger boot problem than First problem.
You need to put, in Bios, every Pcie slot with Auto. After you can try only 1 card and to try Pcie 2.0 and later 3.0.
With Windows and GPU-Z, you can know "Bus Interface" need for yours GTX 690. At the next boot Bios, put the same "Bus Interface" for yours GTX 690.

You will can change Pcie Bus Interface version for your GPU card by:
http://nvidia.custhelp.com/app/answers/detail/a_id/3135/~/geforce-gen3-support-on-x79-platform
it run for Intel C602 chipset and GTX 6xx.

Let's us to know

Best regards
Xavier


----------



## Cogent

Quote:


> Originally Posted by *Wam7*
> 
> The Asus Z9PE-D8 does not have onboard VGA (well mine doesn't), so check you are referring to this motherboard.


I have ASUS Z9PE-D8 WS MBO and it does have onboard GPU. You need special bracket with VGA port that plugs on connector under last PCIe slot.
Quote:


> Originally Posted by *xavierFR*
> 
> Hi,
> 
> I had 2 same problems but i not remember about code error...
> (sorry for my english... I am French)
> 
> First problem.
> From my experience, I build 4 way SLI GTX Titan Kepler. But with Win 10, or Win 2012 rt, and last 5802 Bios, I had a boot problem. 2 cards were not recognized and I had problem with win10 and Win 2012 rt.
> I have use 5701 Bios. Now, I have no boot problem with Win 10 or Win2012 rt.
> 
> GTX 690 is a dual GPU, from Bios and Windows, it is as like 2 way SLI. 2 x GTX 690 is as like 4 way SLI GTX 690, I council you to put 1 card in Slot 1 , and second card in Slot 5.
> 
> Second problem.
> It can be Pcie 2.0 or 3.0 version for yours GTX 690.
> If you have Pcie 2.0 GPU card and use Pcie 3.0 slot, you will have a more bigger boot problem than First problem.
> You need to put, in Bios, every Pcie slot with Auto. After you can try only 1 card and to try Pcie 2.0 and later 3.0.
> With Windows and GPU-Z, you can know "Bus Interface" need for yours GTX 690. At the next boot Bios, put the same "Bus Interface" for yours GTX 690.
> 
> You will can change Pcie Bus Interface version for your GPU card by:
> http://nvidia.custhelp.com/app/answers/detail/a_id/3135/~/geforce-gen3-support-on-x79-platform
> it run for Intel C602 chipset and GTX 6xx.
> 
> Let's us to know
> 
> Best regards
> Xavier


So, you reverted to older BIOS and that solved your problems? I might try that.

I have tried slots 1 and 3, both with single and dual GTX 690's.

About last tip-it doesn't help me since I can't even enter the OS. This problem starts during POST.

I am currently in communicationwith ASUS support and I really hope that they can provide me with PLX rom and vBIOSes for GTX 690.


----------



## Cogent

BIOS 5701 doesn't help. I am getting B2 error again. :/


----------



## xavierFR

Hi,

My experiences, if it can help you&#8230;

-From first problem.

I had 5802 Bios.
In 4 way SLI, Windows was seen 4 GPU card, but 2 GPU was Unknow and not running, and with issue.
I was thinking a issue with Pcie slot 7.
I have use 2 way SLI and 3 way SLI with slot 7 (some time without slot 3 or without slot 5). All was running. It was not hardware issue.
From Internet, I have find to need 5701 Bios, because 5802 Bios not permit 4 way SLI.
Yes, I have downgrade the Bios, from 5802 to 5701. And it run.

-From my second problem.

I had 6 GTX Titan ( 5 now ) but I use only 4 and a AMD HD 7850. Some Gtx Titan was create bad Boot.
At the begining, it was possible to obtain windows boot 2 time, but Windows was not stable. After, it was impossible to obtain Windows boot. The computer was turn OFF.
I have find the Pcie Bus Interface GPU version problem:
http://nvidia.custhelp.com/app/answers/detail/a_id/3135/~/geforce-gen3-support-on-x79-platform
card by card, I have boot, see in Windows with GPU Z software, and test Pcie Bus Interface In Bios, with "Auto", "Pcie2.0", "Pcie3.0". and it was running.
After I have use it:
http://nvidia.custhelp.com/app/answers/detail/a_id/3135/~/geforce-gen3-support-on-x79-platform
I have put all GTX Titan to Pcie3.0, I have boot, put "Pcie3.0" in Bios and it was running.

-Problem 3.

I was buy used GTX Titan. The card was overclocked with waterblock, transform and badly assembled by last owner and before sale.
I had bad issue. In Slot 1, very difficult to obtain windows boot. And I not obtain any information by this way. The computer was turn OFF.
I have use 2 way SLI boot, this GPU card in Slot 1 and in other Slot. By this way it was possible to boot but the GPU card. I was active and unactive (in the Bios or Windows) the GPU card with issue. I have understand a GPU card problem. With and without to use SLI bridge, I have confirme GPU card problem. With and without to active/unactive SLI bridge (some time to remove SLI Bridge, without SLI bridge) and GPU card, and activate/unactie with Windows the card and the SLI, I have confirm hardware GPU card issue.
after investigation, I had short circuit between fan harness GTX Titan and the metal body. The card was very badly assembled by last owner. I have solve the short circuit fan harness and it run.

-Problem 4.

I not have same GTX Titan Kepler. I have 3 original version from EVGA and 1 superclocked from EVGA and 1 GTX Titan original from Gigabyte. From 4 way SLI, only EVGA permit 4 way SLI. Other manufacture permit only 3 way SLI. Then, the manufacture not permit to mix manufacture GPU card mark. For permit 4 way SLI and to mix manufacture mark, for me, it need to NOT USE "NVIDIA control panel", and it need to use "Nvidia Inspector 1973" (it is my version) and to put 4 way SLI code.

-Problem 5

I had big boot issue with a second GTX Titan superclocked. (In any way, the GTX Titan driver run for Original version, Superclocked version and Watercooling original version. it is same driver).
With this second GTX Titan Superclocked version, it was very difficult to obtain windows boot, very slow windows and the computer was turn OFF.
I have try card by card, I have boot, see in Windows with GPU Z software, and test Pcie Bus Interface In Bios, with "Auto", "Pcie2.0", "Pcie3.0". nothing was change.
After I have use it:
http://nvidia.custhelp.com/app/answers/detail/a_id/3135/~/geforce-gen3-support-on-x79-platform
I not obtain switch Pcie2.0 to Pcie3.0. nothing was running. For me the card had a issue, I not have solve the problem and I not have keep this GPU card.

-Problem 6

The problem 5 was create Windows issue. Then, I have install new Win10 and dual boot with Win 2012 rt. Win 10 permit me to find the bigger software limit, and to confirm, detect end solve with Win 2012 rt.
For information, only Win 2012 rt can permit full 4 way SLI performance, not Win 10, but I need win 10 for some software not need full GPU performance. Since Win10, 4 way SLI are not possible, too much complicate and with automatic Win10 upgrade.

-From your card.

Your fisrt problem, the GTX 690 have 2 GPU, it is impossible for you to test 3 way SLI. With 2 GPU GTX 690 cards, you obtain the same of 4 way SLI and you can not obtain 3 way SLI. By this way you can not confirm for sure to need to must use only 3 way sli maximum and by confirm the 4 way SLI limit from motherboard manufacture, 4 way SLI from Nvidia limit, 4 way SLI from windows limit&#8230;

About "I have ASUS Z9PE-D8 WS MBO and it does have onboard GPU. You need special bracket with VGA port that plugs on connector under last PCIe slot.".

I council you to turn OFF this function, or to not put it. use with 1 GTX 690. And to test.

I council you to use Slot 1 and Slot 5. The why&#8230; Slot 1 to Slot 4 come from CPU1, Slot 5 to Slot 7 come from CPU2. to use Slot 1 to 4 with GTX 690, It is as like 4 way SLI in the CPU1. I don't know, but this motherboard permit to obtain 4 way SLI with using the 2 CPU with slot 1 , 3, 5 and 7. You can test Slot 5 with 2 GTX 690 (Slot 1 and Slot 5).

-From your information

"I have tried slots 1 and 3, both with single and dual GTX 690's.".
I understand you have same problem with (only) 1 GTX 690 in slot 1. If my past problem not solve your problem, your problem can come from driver, BIOS card or issue from GTX 690. But same problem with 2 card are very curious.

-From your information

"About last tip-it doesn't help me since I can't even enter the OS. This problem starts during POST."
Do you have change Pcie Bus Interface In Bios, and test with "Auto", "Pcie2.0", "Pcie3.0" ?
Turn off the other Slot. keep only Slot 1.
Can you test with other GPU card (not GTX 690) ?

Your GTX 690 are more old than UEFI Bios... do you know if your GTX 690 run with UEFI Bios ?

What Ram you use?
Do you have official Xeon 2670 or SE for engineer development ?
Do you have put Molex power at the top right of mother board?
And all other Power connector?
What Windows version you use ?
How you have install Windows with GTX 690?
From your first boot, what it was running when you have install Windows?
You can downgrade Bios at the 0311 version. and after to upgrade to 5701 , but not 5802.

With no more information, I not see.
I can only to council you to put first (the most old) driver version , first Bios and first firmware running with mother board, 1 CPU and 1 GTX 690 first (better: a other GPU card). Your first Bios for CPU 2670 Xeon 1 , is 0311 version. I don't know the other version for your GTX Titan and Drviver for Windows.
Else, only to install new Windows 7 64 or Xp 64 (I don't know the GTX 690 OS compatibility driver and OS), with less component. Only 1 memory card, 1 CPU, 1 GPU, 1 Hdd, no more. If you solve your problem, after, it will be possible to Upgrade Windows, Bios, Firmware, Driver&#8230;. Step by Step&#8230; I was making it&#8230;

Here they Speak about B2 Error code
https://rog.asus.com/forum/showthread.php?92442-Q-Code-Error-Code-B2/page2
"The B2 usually means Legacy ROM has an error or Graphic card problem"
i don't know what it is "pulling the trigger " ?

Best regards
Xavier


----------



## Cogent

I am going back to my old CPU's and MBO. I really can't have my main PC offline for such a long time just to achieve nothing.

Please, if anyone who ever had GTX 690 working with Z9PE-D8, please post me instructions and vBIOS for GTX 690.


----------



## WillInFL

Please forgive my ignorance and, before anyone tells me that this is already posted else ware on this site, I don't have time to wade through thousands of posts. I just want to ask a couple of questions that I'm hoping someone will be kind enough to answer without hassling me.

I have spent time searching for the best BIOS settings for the Z9PA-D8 with E5-2697w (v1) CPUs and can't find much in the way of pointed, direct answers. All I ever find are lengthy discussions that seem to theorize on the meaning of life the universe and everything. Why the hell can't someone post a complete listing of the BIOS settings on this board with a good description of how they should be set and why (at least for the main settings that affect performance and stability).

10.jpg 927k .jpg file


----------



## WillInFL

Quote:


> Originally Posted by *Cogent*
> 
> Currentlly, I can't. I tried many times after first two succesfull attempts, but now it only boots with onboard GPU. I trows B2 error every time with GTX 690.


You mention that you're running BIOS version 5802. I have version 5602 which is the highest version available from ASUS. If you didn't typo that and are really running a higher version, can you please tell me where I can find it?


----------



## Cogent

Quote:


> Originally Posted by *WillInFL*
> 
> You mention that you're running BIOS version 5802. I have version 5602 which is the highest version available from ASUS. If you didn't typo that and are really running a higher version, can you please tell me where I can find it?


Here you go, mate: http://dlcdnet.asus.com/pub/ASUS/mb/LGA2011/Z9PE-D8-WS/BIOS/Z9PE-D8-WS-ASUS-5802.zip?_ga=2.147258366.833214835.1516290239-1201954981.1495093625


----------



## WillInFL

Quote:


> Originally Posted by *Cogent*
> 
> Here you go, mate: http://dlcdnet.asus.com/pub/ASUS/mb/LGA2011/Z9PE-D8-WS/BIOS/Z9PE-D8-WS-ASUS-5802.zip?_ga=2.147258366.833214835.1516290239-1201954981.1495093625


OK then. Now I can make fun of myself: I made a mistake by coming to this forum since it's for the Z9P*E*-D8 and I actually have the Z9P*A*-D8.


----------



## karakarga07

*I have managed to set to PCIex 3.0 speed at 3rd slot!*

Hi to all,

My Asus Z9PE-D8 WS mainboard sets succesfully to PCI Express 3.0 speed from 3rd PCI express slot with Asus manufactured GTX 780Ti (orignal nVidia single back blower cooler model) but this mainboard can not run the same graphics card fitted to 1st slot, which is interesting!

I have tried one Zotac 670 4GB base model (single back blower again) with "no success" at 1st and 3rd slots.
I have also tried one Zotac 680 2GB base model (single back blower again) with "no success" at 1st and 3rd slots again.

The problem doesn't seem like by the test software, GPU-Z, HWiNFO64, Aida and SiSoft Sandra all informed the same results like each other every time! 

There is definitely a compatibility issue with majority of cards! I have currently only 1 Xeon E5-2658 CPU on it, therefore I can not managed to try on slots 4 to 7.


----------



## celemine1gig

karakarga07 said:


> Hi to all,
> 
> My Asus Z9PE-D8 WS mainboard sets succesfully to PCI Express 3.0 speed from 3rd PCI express slot with Asus manufactured GTX 780Ti (orignal nVidia single back blower cooler model) but this mainboard can not run the same graphics card fitted to 1st slot, which is interesting!
> 
> I have tried one Zotac 670 4GB base model (single back blower again) with "no success" at 1st and 3rd slots.
> I have also tried one Zotac 680 2GB base model (single back blower again) with "no success" at 1st and 3rd slots again.
> 
> The problem doesn't seem like by the test software, GPU-Z, HWiNFO64, Aida and SiSoft Sandra all informed the same results like each other every time!
> 
> There is definitely a compatibility issue with majority of cards! I have currently only 1 Xeon E5-2658 CPU on it, therefore I can not managed to try on slots 4 to 7.


I have this board for a while now and tested with several configurations (single CPU vs. dual CPU, different GPUs and other cards, different types of RAM) and one thing is clear:
If you want to operate this thing at a stable working point, you have to read through all the Intel Errata sheets VERY CAREFULLY and draw your conclusions from the problems mentioned there.

*For your problem with the cards not negotiating to Gen3, there are several known issues. BIOS also has switches for workarounds.*
However, in case of my Geforce 980Ti in Slot 4, I had to force the slot to Gen3, otherwise it would never automatically train to a Gen3 PCIe link. There are also BIOS switches to accomplish that.

Besides that, this board is a real ***** when it comes to ECC REG memory in 2x 4x4GB configurations, when using dual CPUs. Seems to be due to an known Erratum, that Asus does not seem to care about. Function behind this is "Staggered Refresh" for Memory, which Asus does not seem to have enabled and there is no way to do so with BIOS methods. Maybe with EFI shell scripts, but I simply gave up due to a lack of time.
Sent in a board once for RMA, due to this problem. Instead of investigating the underlying cause, I just got a brand new board back from Asus. Which of course also has the issue. 

Also the board is missing ME and Microcode updates for severe security flaws. It really is a mess and eating up a lot of time to work with this thing. Really a pity.


----------



## Wam7

Yes this motherboard is one where you keep things as simple as possible. I use it with a passive GT720 GPU, and 1 SSD.
With 2 x 2680 V2 I can overclock on the BClk to 109 which is a lot better than the 104 I got with a pair of 2670's.


----------



## tinyhawkus

Hey All,

Apologies, but thread search didn't seem to be working at all.

My Z9PE-D8WS is running really well, the only problem I have, is the 2 onboard NICs seem to be causing havoc with software
that is locked to a hostid. Some are working fine, others are not working at all.

Normally I'd hop right in and disable at least one of them, on other systems. However, I can't seem to find anything in the settings.
There's an option for oprom being either PXE, or scsi, disabled, but nothing to actually disable them altogether.
Have I completely missed something, or is this one of the only boards I've seen without the option to disable onboard networking?

Cheers

Lewie


----------



## Cogent

I haven't report back. I managed to do it. It required multiple methods, but my GTX690's were finally running in quad SLI configuration.

@ tinyhawkus: I thinks I saw the option to disable onboard NIC's. But, why would you want to do that? Those are Intel NIC's which should be compatible with everything.

Question for anyone who wish to respond:

Is someone here running this board with ES CPU's? I am aiming at pair of QEEY's which are ES versions of 2697 V2-C0 stepping. Price seems ok, but I want to be 100% sure it can work with this board and latest BIOS.


----------



## lowfat

Has anyone ever got an ASMB6-iKVM module to work on the Z9PE-D8 WS? I install the card. The POST LCD counts down from about 80 down to 00. Then it just sits there.


EDIT:




featheredtar said:


> I am using the latest BIOS. I have an update. Overnight the computer booted up!
> 
> Basically now whenever I restart it it takes 10-15 minutes of the error screen staying at 0 after the ASMB6-iKVM chip initializes, and then the mobo moves on to CSM initialization (diagnostic code 79). This process takes around 10-15 minutes. It all seems a bit excessive - what can I do to fix this?




15 minutes?!?!


Don't suppose you ever resolved this?


EDIT: 1+ hours and it still hasn't POSTed yet. 
EDIT2: It posts eventually. But it doesn't seem to detect the ASMB6. I try to flash the ASMB6 firmware but it doesn't get listed in the flashing tool.


----------



## Daarn

HI
I try to enable VT-d on my z9pe-8D ws motherboard. but everytime I press F10 and restarts and enter bios agian it disable again 
My cpu are https://ark.intel.com/products/6458...E5-2680-20M-Cache-2_70-GHz-8_00-GTs-Intel-QPI
Any tips and ide?

Have try to disable Marvell sata controller but still the same.


----------



## BlackArchon

Daarn said:


> HI
> I try to enable VT-d on my z9pe-8D ws motherboard. but everytime I press F10 and restarts and enter bios agian it disable again
> My cpu are https://ark.intel.com/products/6458...E5-2680-20M-Cache-2_70-GHz-8_00-GTs-Intel-QPI
> Any tips and ide?
> 
> Have try to disable Marvell sata controller but still the same.


VT-d is broken on C1 stepping Sandy Bridge-EP CPUs. Most probably you have such a CPU. If you really need VT-d, either get a C2 stepping Sandy Bridge-EP or any Ivy Bridge-EP processor.


----------



## Daarn

BlackArchon said:


> VT-d is broken on C1 stepping Sandy Bridge-EP CPUs. Most probably you have such a CPU. If you really need VT-d, either get a C2 stepping Sandy Bridge-EP or any Ivy Bridge-EP processor.


OKej looks like I have C1

the this cpu should work
https://www.ebay.com/itm/Intel-Xeon...867380?hash=item3f9aab7474:g:wKQAAOSwaaJa4DKi


----------



## BlackArchon

Daarn said:


> OKej looks like I have C1
> 
> the this cpu should work
> https://www.ebay.com/itm/Intel-Xeon...867380?hash=item3f9aab7474:g:wKQAAOSwaaJa4DKi


This is an Ivy Bridge-EP CPU, so it has working VT-d. Don't forget to flash the latest BIOS on your mainboard before installing this CPU.


----------



## MagikSun

*LAN Port issue in Windows 10*

Hiho, 

I've recently acquired an Asus Z9PE D8 WS mobo. In Windows 10 only one of the two LAN Ports work. The other port has a steady orange light and it's not detected in device manager. It's not a big deal as the other port works fine but itd be nice to figure it out. 

Has anyone else had this problem with windows 10? It's the Intel® 82574L dual Gigabit ports and I've tried downloading drivers from intel and reinstalling to no avail. 

My next step will be to try a Windows 7 install to see if it's a Windows 10 driver thing or if the port is actually faulty, but if anyone else has had this experience it'd be helpful to know.


----------



## Wam7

I'm running Windows 10 (one of the earlier versions) and both ports are working fine on mine so I doubt it is a Windows 10 issue but you never know.


----------



## Daarn

BlackArchon said:


> This is an Ivy Bridge-EP CPU, so it has working VT-d. Don't forget to flash the latest BIOS on your mainboard before installing this CPU.


Thx for the help.
I have come over one par of Xeon X5650 SLBV3 for free
Any Info if this one works with VT-d on this board?


----------



## Wam7

Daarn said:


> Thx for the help.
> I have come over one par of Xeon X5650 SLBV3 for free
> Any Info if this one works with VT-d on this board?


Xeon X5650 will not work at all in this board. Those are X58 socket 1366 CPU's this board need X79 Sandybride/Ivybridge Socket 2011 CPU's.


----------



## meathelix

Wam7 said:


> Yes this motherboard is one where you keep things as simple as possible. I use it with a passive GT720 GPU, and 1 SSD.
> With 2 x 2680 V2 I can overclock on the BClk to 109 which is a lot better than the 104 I got with a pair of 2670's.


Just wondering, what settings are you using to get your's to 109? I am lucky to get over 105. 

I have two 2680's V2 and ECC memory at 1600mhz 64gig of it. 

What voltage are you running at and your other settings? Thanks


----------



## Wam7

meathelix said:


> Just wondering, what settings are you using to get your's to 109? I am lucky to get over 105.
> 
> I have two 2680's V2 and ECC memory at 1600mhz 64gig of it.
> 
> What voltage are you running at and your other settings? Thanks


Sorry for the very late reply, have been out of my country for a while. For 109 Mhz I did nothing special apart from set the CPU to performance mode and set the BCLK. I'm running just 2 sticks of 8Gb (16Gb in total) which will help with overclocking. I could boot at 111Mhz but rock solid stability was at 109 Mhz


----------



## Blindsay

Hey all,

Unless i am mistaken you cannot (without mods) boot from an nvme drive on this board correct (if it is attached to a m.2 > pcie adater)?

My question is though will it show up as a secondary drive?


----------



## Bigsulls114

Hi guys, I tried increasing the BLK to 106 on this board and changed other bios settings which resulted in a non booting system. I then tried to reset the bios via the jumpers and now the system will not boot. The Leds are on but the Qled is not showing anything and the fans spin for a second but nothing else. I ordered a new bios chip and its the same behavior. I think the new bios chip is faulty, but not ruling out the board is now nackered which would be surprising. Any advice would be greatly appreciated. 

P.S I have tried swapping cpus, running one cpu and changing ram etc all with the same behaviour with no post. Also tried a different PSU..Same deal..


----------



## Wam7

It would be strange that overclocking to 106 Mhz would cause something terminal to the motherboard. What CPU's are you using?

That being said this board can be a ***** to revive after a too high overclock. I've gone from 2670's through a variety of cpu's and now have the best available for this board in a pair of E5-2697 V2's. These are running at 113 Mhz but in exactly the same board a pair of 2670's would not go above 105 Mhz and some 2690's wouldn't go above 104 Mhz. 

In the case with all CPU's and going too high it would eventually just not post. Most times I could get it to post by pulling the power cable out the back and leaving it for a couple hours. I could then get in the bios and adjust accordingly. On a few occasions the only way to get it to post was to remove both CPU's from the board and then leave it un-powered and without the CMOS battery for several hours. I would try that and see what happens.


----------



## kuronekocfu

Hello guys. I have a Z9PA-D8 (not Z9PE-D8 WS). I have been working on it with 2x2620v1 and 8x4GB DDR3 Non-ECC for 5 years now. Today I have finally received two 2680v2 CPUs. After installing them I got the "bc" error on the motherboard and that's how I ended up in this thread. This error came up 5 years ago when people were trying to use dual v2 CPUs on the Z9PE-D8 WS with Non-ECC memory. People reported that the system boots fine with just one CPU and it is resolved by using ECC memory. Then Asus in a firmware update for the Z9PE-D8 WS addressed the issue and after that people could use dual v2 CPUs with Non-ECC memory just fine. That was back in 2013 and the conversation about it starts at page 82 of this thread. I'm running the latest firmware on my motherboard (year 2015) but it seems like I'm still getting that error and thus the problem seems to not have been addressed on the Z9PA-D8. I don't mind changing my memory to ECC if I knew that's the problem indeed although it's sad that I have to do this upgrade just for something that can be fixed on a firmware. Given that Asus hasn't updated these motherboard for 3 years now it's unlikely they would bother to. Did other people have any similar problem and how did you resolve it?
Thanks.


----------



## MustangSVT

Do any of you guys use the optical audio on this thing? I saw in the manual/online it says it has DTS Connect support. Do I just need to download the audio drivers for this? Reason I ask is cause on ASUS site, they only list drivers for up to Win8.1 and I'm on Win10.


----------



## VitOne

Dear All,

I am back to this forum. After year of usage, my Z9PE-D8 started to give me some problems. It is very difficult to boot. Sometimes I have to restart the Workstation several times before it boots.

The motherboard gives error d7. Please notice that after it starts, everything runs smoothly as usual. So the problem that I have is that the PC won't start randomly for a number of times, than it will boot and run fine until I turn it off.

Error d7 is officially listed on the manual as: "no console input devices are found".

I changed mouse, keyboard, other USB devices (I removed them all) but nothing changes. The system simply refuses to boot for a number of times and then, leaving all as it is and forcing reboots, at a certain point it will "magically" boot without errors.

During usage system is very stable. I can run 48hrs or longer analysis (COMSOL, Ansys, etc) without problems and smoothly.

I tried to search for this error code without luck, I'll try to ask Asus directly. It seems that it can be related to PSU problems (12V connector) but, as I said, after boot I can run 100% CPU for hours without problems.

I didn't change a component in the last years (this workstation is from 2014, I updated RAM and moved some components in 2016 probably) on this built and I never had particular problems.

Old video: 



My setup is:
2 x Xeon 2687 v2
8 x 8Gb hmt31gr7cfr4a-h9 (ECC RAM)
Quadro K6000 (tried to change it, nothing changed)
Sound Blaster ZxR (tried to remove it, nothing changed)

Enermax Revolution 85+ 1250 Watt (very old PSU)
APC Smart-UPS - SMC1500I (or similar unit, I am not 100% sure about the exact model)

Does anybody have similar experience with this error code? Thank you!


----------



## NoS_rb26

Hello, 

I'm new to the forum and I'm coming here because I need help. I bought 1 month ago a motherboard z9pe d8 ws with two e5 2650. 
I also have a corsair rm1000x and a 780Ti. However when I start the motherboard, it blocks on code b1 and does not display anything on the screen. 
I disassemble the cpu and I realize that one of the sockets (socket of cpu 2) has a bent pin. 
I straighten it and restart the card, still nothing. I test with only one cpu on socket 1, the same always the code b1. 
In CPU 1 configuration, on ram slots C1 and C2 I block on code b1 otherwise it is code b0 which is displayed. 
I thought it is the ram because I don't have a ddr3 ecc, but a stick like this : https://www.ldlc.com/fiche/PB00164466.html. I really tried everything with many clear cmos etc. I don't see any other solution. 
I think the problem comes from the ram but I'm not sure. I have done a lot of tests and still can't fix it. Thank you very much, cordially.


----------



## MasterHome

Hello, All!
Corrected BIOS 5802 to select the operating mode of PCI-E 2a. Updated CPU microcodes. Added bootable from NVMe. Only flash the programmer. https://yadi.sk/d/JcCWqphE1UXdGQ


----------



## VitOne

VitOne said:


> Dear All,
> 
> I am back to this forum. After year of usage, my Z9PE-D8 started to give me some problems. It is very difficult to boot. Sometimes I have to restart the Workstation several times before it boots.
> 
> The motherboard gives error d7. Please notice that after it starts, everything runs smoothly as usual. So the problem that I have is that the PC won't start randomly for a number of times, than it will boot and run fine until I turn it off.
> 
> Error d7 is officially listed on the manual as: "no console input devices are found".
> 
> I changed mouse, keyboard, other USB devices (I removed them all) but nothing changes. The system simply refuses to boot for a number of times and then, leaving all as it is and forcing reboots, at a certain point it will "magically" boot without errors.
> 
> During usage system is very stable. I can run 48hrs or longer analysis (COMSOL, Ansys, etc) without problems and smoothly.
> 
> I tried to search for this error code without luck, I'll try to ask Asus directly. It seems that it can be related to PSU problems (12V connector) but, as I said, after boot I can run 100% CPU for hours without problems.
> 
> I didn't change a component in the last years (this workstation is from 2014, I updated RAM and moved some components in 2016 probably) on this built and I never had particular problems.
> 
> Old video: https://www.youtube.com/watch?v=QxUsfTqEnyA
> My setup is:
> 2 x Xeon 2687 v2
> 8 x 8Gb hmt31gr7cfr4a-h9 (ECC RAM)
> Quadro K6000 (tried to change it, nothing changed)
> Sound Blaster ZxR (tried to remove it, nothing changed)
> 
> Enermax Revolution 85+ 1250 Watt (very old PSU)
> APC Smart-UPS - SMC1500I (or similar unit, I am not 100% sure about the exact model)
> 
> Does anybody have similar experience with this error code? Thank you!


I would like to inform you that, after months, I tried to solve the problem changing: PSU, VGA, CPUs, RAM, HHD.
The final solution was to change the BIOS chip. Please notice that I flashed the old BIOS several times thinking that that could be the problem, but nothing changed.

So, in case you are experiencing D7 message, you could try to replace your BIOS chip.

Best regards to all of you!


----------



## Nasmo

MasterHome said:


> Hello, All!
> Corrected BIOS 5802 to select the operating mode of PCI-E 2a. Updated CPU microcodes. Added bootable from NVMe. Only flash the programmer. https://yadi.sk/d/JcCWqphE1UXdGQ


Im gonna try this when I get my bios flasher in the mail. Hopefully I can use my top pci-e slot again @ x16 3.0


----------



## Nasmo

Nasmo said:


> Im gonna try this when I get my bios flasher in the mail. Hopefully I can use my top pci-e slot again @ x16 3.0


Unfortunately, I am still stuck at pcie 3.0 x16 x4 after the bios install.


----------



## MasterHome

Nasmo said:


> Unfortunately, I am still stuck at pcie 3.0 x16 x4 after the bios install.


Port 2A Link Speed


----------



## NoS_rb26

Hello, does anyone can't help me with my problem ?


----------



## 10Dev

NoS_rb26 said:


> Hello, does anyone can't help me with my problem ?


I just started using this motherboard a few months ago. There are a lot of pages in this thread that are full of information.

Clear CMOS is a huge activity on even small changes with this board. Sometimes you have to remove the battery for the clear to work. And sometimes when you think it won't boot it decides eventually to start up so always give it at least 2 minutes.

Make sure your RAM is on the QVL list.

The board is also very picky about different GPU cards and which slot they are in and sometimes you just have to change the slot to get to BIOS and then you can just change back.

Although I have not tried it, (yet) the BIOS is a plug-in chip and there are a few people offering a new BIOS with any version you want for about $20 on ebay.


----------



## 10Dev

NoS_rb26 said:


> Hello,
> I disassemble the cpu and I realize that one of the sockets (socket of cpu 2) has a bent pin.
> .


A few years ago, in a very stupid moment I accidentally bent pins in a socket 1150 mobo.

They are very hard to get straight. And very easy to miss seeing ones that are still bent.

You need a very strong magnifying glass combined with a very strong lamp. 

Then it is important to rotate the motherboard 90 degrees a bunch of times due to the different angles of the pins. It took me about 10 hours with many breaks to finally be satisfied I had repaired all the pins...

You might still have other bent pins and even might want to check the "good" socket again.


----------



## 10Dev

*Three Rank DIMM*

A DDR3 24gb HP 3 Rank RDIMM is starting to appear on ebay at a decent price.

This was used in HP servers with E5 V2 and so maybe it would work in the ASUS?

So, has anyone tried 4 of these in the Z9PE-D8?

The HP docs say all 16 slots in their server must be populated for these particular RDIMMs to work, but just seems like a sales thing.

The very detailed Fujitsu memory doc makes no reference of a 3 Rank RDIMM

https://sp.ts.fujitsu.com/dmsp/Publications/public/wp-sandy-bridge-ep-memory-performance-ww-en.pdf

So I am inclined to think that without specific BIOS support these affordable high density modules are unlikely to work...


----------



## 10Dev

*Some misc notes on Z9PE-D8 in the year 2020*

I recently ended up with this mobo as an affordable transition to something more modern.

The current cost of used components makes this less costly than a Ryzen 3800x with similar performance via two E5-2690 V2

The current advantage with this older mobo is the 256GB DDR3 RAM which is still very expensive in DDR4 which also needs an expensive Threadripper mobo and a Threadripper CPU.

Misc Notes for Personal Computer or Workstation scenarios:

- in 2020, the Ryzen is probably a better starting point if starting from scratch, and for sure if you don't have any "Tinkerer" in your genes. The E5 V3s are finally showing up cheap in surplus but that needs a V3/V4 mobo and DDR4 RAM. The V3 but not the V4 has a "Haswell Hack" overclock for running all cores in Turbo but given the price hit on DDR4, the Ryzen is still a better choice for most people, although if your needs hit a Threadripper choice, there is an agonizing "value of money" versus "Future Proof" consideration there.

- the 32 GB DDR3 RDIMM has price and availability that fluctuates a lot so much patience is required to get 8 of them at a decent price

- There is ZERO practical benefit to the almost non-existent overclocking of E5 dual CPUs

- when selecting two CPUs for personal use, you want clock speed over 3Ghz and a TDP rating of 130 watts or more. The Xeon will not exceed it's power rating so that means you want the 150 watt 2687 or the handful of 130 watt units

- without a modded BIOS, you must boot from a SSD which is not horrible and you can use multiple NVMe Drives in the expansion slots. You won't find an affordable adapter that understands PCIe bifurcatiuon but you have so many slots that 2 or three $10 NVMe slot adapters should meet most WS needs. I am using the ADATA XPG SX-8200 Pro which meets its 3000 MB/sec spec.

- GPU slot placement is sensitive and anything powerful these days kills 2 slots so one of the original scenarios of 8 GPUs is no longer useful.

- most of the enormous list of issues in this thread goes away with the latest BIOS available combined with the latest Windows 10 drivers and the result is very stable. By "latest drivers" that does not mean the default Microsoft drivers. A lot of Intel drivers don't default right etc. just list all the hardware and get various chip manufacturer drivers where often something dated 2011, 2014 etc will be "latest" compered to Windows default. Very noticeable difference in stability when making this extra effort.

- you will probably want to add a USB 3 card, since the 4 included ports is low for everyday usage as a PC.

- you will want to strongly consider your personal comfort with disabling Spectre and Meltdown patches for older Intel CPUs since for many people they just kill some performance for almost no security improvement. I personally don't want a useless perf hit on my 20 real cores!

- when initially setting up the hardware, you might need to use a full CMOS Reset far more often than you are used to doing. But it works for mysterious non booting for simple things like RAM re-config and changing GPU slot etc.

- also, on many hardware changes some sort of test seems to get invoked and the board can seem like it won't boot, but just give it an extra 2 minutes!

- compared to a single CPU consumer mobo, once it is all working, the hardware seems to flow extra fast and smooth probably due to the huge increase in PCIe lanes.

- my initial experiments with multiple GPUs suggest things will work better when using slots connected to a single CPU, so I have my GPUs on the first CPU and all the NVMe drives on the second CPU.


----------



## fbb

Hi there. I recently purchased a used computer that has a Z9PE-D8 WS mobo and am in desperate need of some help.

I made a bunch of adjustments to this system and it was working fine except that at a certain point the fans on the cpu's started running really fast and loud (after I replaced the thermal paste on the cpu's) so I restarted my machine to look at the fan control settings in the bios. They were set to "general", which I took to be a kind of "auto". I then noticed that the mobo was reporting higher temps for the cpus (ranging from 45 to 55) than I had seen previously in speccy. This seemed odd as with the fans running so fast they were noticeably cooling down the space around the computer but it would explain the high fan speeds. I wondered if maybe this was a faulty reading, or if my application of thermal paste could have been bad somehow, but in the end, I decided to apply the bios option to "reset to optimized defaults" and now I don't see any kind of loading information on the monitor at all - nothing but black, though the computer is clearly powered up.

*Things I've noticed or done to solve the problem:*

I initially restarted the computer via the power button on the case a number of times.
I then tried to reset the bios using the "ASUS CrashFree BIOS 3" method (Managing And Updating Your Bios; Asus Crashfree Bios 3 - Asus Z9PE-D8 WS Manual [Page 71] | ManualsLib) but I suspect I did it incorrectly. All I did was dump the Bupdater_130.EXE and Z9PE-D8-WS-ASUS-5802.CAP files on a usb stick which I had previously setup as a windows install drive and plugged that in, expecting the computer to "automatically recover the BIOS" as per the manual's description of what should happen. I used a usb drive that was setup for installing windows because it's the only reliable one I have, and I didn't clear the windows installation media from it first because I didn't want to have to remake it and reasoned that if this method worked the computer would simply recognize the relevant files and use them appropriately, but there are a number of points here where my assumptions could be wrong. I waited several minutes with the usb in the computer and nothing appearing on the screen before restarting it, but I worry I may have done so too soon.
After that, I did all I could to reset the cmos, including the jumper method and pulling the cmos battery, discharging the power button for 10 seconds and then leaving the system with the cmos battery out for several hours before putting it back in and powering up again.
Throughout my numerous restarts I also tried moving my gpu (titan x) to a few different pcie slots, putting in an older gpu (gtx 570) instead, and I removed all other peripherals and hard-drives, and moved to using just one monitor connected through DVI.
After doing all that, I noticed the numbers displayed on the mobo near its power and reset buttons. On startup, these numbers - the Q-codes - run through a bunch of numbers but finally gets stuck on "71", which, according to the mobo manual (Asus Z9PE-D8 WS Manual (Page 46 of 226) | ManualsLib) is "PCH DXE SMM initialization is started". I have no idea what that might mean.
*My computer's specs:*
OS
Windows 10 Pro 64-bit
CPU
Intel Xeon E5 2697 v2 @ 2.70GHz 38 °C
Ivy Bridge-EP/EX 22nm Technology
Intel Xeon E5 2697 v2 @ 2.70GHz 36 °C
Ivy Bridge-EP/EX 22nm Technology
RAM
64.0GB DDR3 @ 936MHz (13-13-13-32)
Motherboard
ASUSTeK COMPUTER INC. Z9PE-D8 WS (Socket-R 2011) 39 °C
Graphics
BenQ EW2420 ([email protected])
BenQ EW2420 ([email protected])
GT2201 ([email protected])
BenQ EW2420 ([email protected])
4095MB NVIDIA GeForce GTX TITAN X (EVGA) 35 °C
Storage
500GB WD Blue SSD (not sure about the model number but I just bought it new last night. This has the windows install on it)
500GB WD Black M.2 NVME drive attached via a pcie adapter (couldn't use this as a boot drive, but it works once windows is loaded. Also brand new.)
1863GB Seagate ST2000DM001-1CH164 (SATA ) 37 °C
7452GB Seagate ST8000DM004-2CX188 (SATA ) 36 °C
1397GB Seagate ST31500341AS (SATA ) 43 °C
931GB Seagate ST31000528AS (SATA ) 36 °C
Optical Drives
HL-DT-ST BD-RE WH16NS40
PSU
1200 watt 80plus Gold

Please help if you can.


----------



## Empherialy

fbb said:


> Hi there. I recently purchased a used computer that has a Z9PE-D8 WS mobo and am in desperate need of some help.
> 
> I made a bunch of adjustments to this system and it was working fine except that at a certain point the fans on the cpu's started running really fast and loud (after I replaced the thermal paste on the cpu's) so I restarted my machine to look at the fan control settings in the bios. They were set to "general", which I took to be a kind of "auto". I then noticed that the mobo was reporting higher temps for the cpus (ranging from 45 to 55) than I had seen previously in speccy. This seemed odd as with the fans running so fast they were noticeably cooling down the space around the computer but it would explain the high fan speeds. I wondered if maybe this was a faulty reading, or if my application of thermal paste could have been bad somehow, but in the end, I decided to apply the bios option to "reset to optimized defaults" and now I don't see any kind of loading information on the monitor at all - nothing but black, though the computer is clearly powered up.
> 
> *Things I've noticed or done to solve the problem:*
> 
> I initially restarted the computer via the power button on the case a number of times.
> I then tried to reset the bios using the "ASUS CrashFree BIOS 3" method (Managing And Updating Your Bios; Asus Crashfree Bios 3 - Asus Z9PE-D8 WS Manual [Page 71] | ManualsLib) but I suspect I did it incorrectly. All I did was dump the Bupdater_130.EXE and Z9PE-D8-WS-ASUS-5802.CAP files on a usb stick which I had previously setup as a windows install drive and plugged that in, expecting the computer to "automatically recover the BIOS" as per the manual's description of what should happen. I used a usb drive that was setup for installing windows because it's the only reliable one I have, and I didn't clear the windows installation media from it first because I didn't want to have to remake it and reasoned that if this method worked the computer would simply recognize the relevant files and use them appropriately, but there are a number of points here where my assumptions could be wrong. I waited several minutes with the usb in the computer and nothing appearing on the screen before restarting it, but I worry I may have done so too soon.
> After that, I did all I could to reset the cmos, including the jumper method and pulling the cmos battery, discharging the power button for 10 seconds and then leaving the system with the cmos battery out for several hours before putting it back in and powering up again.
> Throughout my numerous restarts I also tried moving my gpu (titan x) to a few different pcie slots, putting in an older gpu (gtx 570) instead, and I removed all other peripherals and hard-drives, and moved to using just one monitor connected through DVI.
> After doing all that, I noticed the numbers displayed on the mobo near its power and reset buttons. On startup, these numbers - the Q-codes - run through a bunch of numbers but finally gets stuck on "71", which, according to the mobo manual (Asus Z9PE-D8 WS Manual (Page 46 of 226) | ManualsLib) is "PCH DXE SMM initialization is started". I have no idea what that might mean.
> *My computer's specs:*
> OS
> Windows 10 Pro 64-bit
> CPU
> Intel Xeon E5 2697 v2 @ 2.70GHz 38 °C
> Ivy Bridge-EP/EX 22nm Technology
> Intel Xeon E5 2697 v2 @ 2.70GHz 36 °C
> Ivy Bridge-EP/EX 22nm Technology
> RAM
> 64.0GB DDR3 @ 936MHz (13-13-13-32)
> Motherboard
> ASUSTeK COMPUTER INC. Z9PE-D8 WS (Socket-R 2011) 39 °C
> Graphics
> BenQ EW2420 ([email protected])
> BenQ EW2420 ([email protected])
> GT2201 ([email protected])
> BenQ EW2420 ([email protected])
> 4095MB NVIDIA GeForce GTX TITAN X (EVGA) 35 °C
> Storage
> 500GB WD Blue SSD (not sure about the model number but I just bought it new last night. This has the windows install on it)
> 500GB WD Black M.2 NVME drive attached via a pcie adapter (couldn't use this as a boot drive, but it works once windows is loaded. Also brand new.)
> 1863GB Seagate ST2000DM001-1CH164 (SATA ) 37 °C
> 7452GB Seagate ST8000DM004-2CX188 (SATA ) 36 °C
> 1397GB Seagate ST31500341AS (SATA ) 43 °C
> 931GB Seagate ST31000528AS (SATA ) 36 °C
> Optical Drives
> HL-DT-ST BD-RE WH16NS40
> PSU
> 1200 watt 80plus Gold
> 
> Please help if you can.


----------



## Zogge

Reseating both cpus and all dimms helped me when I had a hard lock on the server when I installed it all and tried to OC. Since I stopped the block oc and run everything standard, it is stable as a rock. I have the Z9PE-D8 WS as well as the Z9PE-D16, both working 100% still.


----------



## RomaKoks

10Dev said:


> *Some misc notes on Z9PE-D8 in the year 2020*
> 
> I recently ended up with this mobo as an affordable transition to something more modern.
> 
> The current cost of used components makes this less costly than a Ryzen 3800x with similar performance via two E5-2690 V2
> 
> The current advantage with this older mobo is the 256GB DDR3 RAM which is still very expensive in DDR4 which also needs an expensive Threadripper mobo and a Threadripper CPU.
> 
> Misc Notes for Personal Computer or Workstation scenarios:
> 
> - in 2020, the Ryzen is probably a better starting point if starting from scratch, and for sure if you don't have any "Tinkerer" in your genes. The E5 V3s are finally showing up cheap in surplus but that needs a V3/V4 mobo and DDR4 RAM. The V3 but not the V4 has a "Haswell Hack" overclock for running all cores in Turbo but given the price hit on DDR4, the Ryzen is still a better choice for most people, although if your needs hit a Threadripper choice, there is an agonizing "value of money" versus "Future Proof" consideration there.
> 
> - the 32 GB DDR3 RDIMM has price and availability that fluctuates a lot so much patience is required to get 8 of them at a decent price
> 
> - There is ZERO practical benefit to the almost non-existent overclocking of E5 dual CPUs
> 
> - when selecting two CPUs for personal use, you want clock speed over 3Ghz and a TDP rating of 130 watts or more. The Xeon will not exceed it's power rating so that means you want the 150 watt 2687 or the handful of 130 watt units
> 
> - without a modded BIOS, you must boot from a SSD which is not horrible and you can use multiple NVMe Drives in the expansion slots. You won't find an affordable adapter that understands PCIe bifurcatiuon but you have so many slots that 2 or three $10 NVMe slot adapters should meet most WS needs. I am using the ADATA XPG SX-8200 Pro which meets its 3000 MB/sec spec.
> 
> - GPU slot placement is sensitive and anything powerful these days kills 2 slots so one of the original scenarios of 8 GPUs is no longer useful.
> 
> - most of the enormous list of issues in this thread goes away with the latest BIOS available combined with the latest Windows 10 drivers and the result is very stable. By "latest drivers" that does not mean the default Microsoft drivers. A lot of Intel drivers don't default right etc. just list all the hardware and get various chip manufacturer drivers where often something dated 2011, 2014 etc will be "latest" compered to Windows default. Very noticeable difference in stability when making this extra effort.
> 
> - you will probably want to add a USB 3 card, since the 4 included ports is low for everyday usage as a PC.
> 
> - you will want to strongly consider your personal comfort with disabling Spectre and Meltdown patches for older Intel CPUs since for many people they just kill some performance for almost no security improvement. I personally don't want a useless perf hit on my 20 real cores!
> 
> - when initially setting up the hardware, you might need to use a full CMOS Reset far more often than you are used to doing. But it works for mysterious non booting for simple things like RAM re-config and changing GPU slot etc.
> 
> - also, on many hardware changes some sort of test seems to get invoked and the board can seem like it won't boot, but just give it an extra 2 minutes!
> 
> - compared to a single CPU consumer mobo, once it is all working, the hardware seems to flow extra fast and smooth probably due to the huge increase in PCIe lanes.
> 
> - my initial experiments with multiple GPUs suggest things will work better when using slots connected to a single CPU, so I have my GPUs on the first CPU and all the NVMe drives on the second CPU.


Could you please provide a link with modified bios (and some installation instruction) which makes nvme boot possible? All related links I've seen are dead...


----------



## Zogge

RomaKoks, I bought my chip from :








BIOS CHIP ASUS z9pe-d8 WS, at3gc-i, at 3 IONT-I, at 3 IONT-I DELUXE, at3n7a-i,... | eBay


ASUS AT3IONT-I DELUXE. ASUS AT3GC-I. ASUS AT3IONT-I. ASUS AT3N7A-I. ASUS Z9PE-D8 WS. DIP-8 Chip DIP-Ausheber PLCC-32 Chip PLCC-Zange CR2032 Batterie PLCC-32 Sockel. (oder auf Wunsch mit einer anderen Version, die der Käufer mir mitteilt).



www.ebay.com





With this question:



 

Hi

Is it possible to mail you a custom bios that you flash before you send it ?

It is a .bin file (with corrections in bios for PCI gen 3, microcode and nvme support) for an asus z9pe-d8 ws mainboard on bios 5802.

(Z9PE-D8-WS-ASUS-5802_nvme4_cpumc_dev_pci-e2a.bin)

Best regards,​


Then I got a reply that he could use it and then he sent me the custom bios programmed on the bios chip which I just swapped and it worked perfectly.

I have the custom bios file saved if you would need it.

PS. It works with Asus Hyper x16 M.2 card with 4 NVMEs as well as the supermicro AOC-SHG3-4M2P card with 4 drives. I have at total of 12 NVME drives in my Z9PE-D8 WS board, fully bootable and usable by the system with full speed. They will show up as PATA: in the bios boot section but once windows is installed or similar you will have the option to boot from them through UEFI.


----------



## RomaKoks

Thanks for reply! Yes the file could be very helpful! I also have z9pe-d16 10g dual, suppose it will have similar problem, if you have some compatible BIOS file for it - it will be great!


----------



## Zogge

Bios


----------



## andt2022

Mine is stuck on Bios Splash Screen and freeze with Q-Code 0d, when i try to get the Bios or EzFlash i will bekame a simple blinking cursor can anyone help me?
Lastest Bios is installed.
2x E5-2760
8x4GB(32GB Ram)
GT1030

All devices are allready disconected


----------



## Zogge

1. Try disconnecting sata drives (already did) 
2. If no success reseat dimms
3. As last resort reseat cpu


----------



## giorg

I am having problems with my Asus Z9PE-D8 WS motherboard and the ASMB6-iKVM IPMI. The latter works when the system is on but doesn't seem to work when the system has shutdown. I have set static LAN IPv4 ip for ASMB6 but it doesn't even ping when the system is turned off, neither does the BMC_LED1 blink. Am I doing something wrong ? Does anyone have a clue what to do ?

I solved the problem, it was probably due to router misconfiguration.


----------



## eZwa306

I recently bought this motherboard used, along with two Xeon CPUs and 32 GB ECC RAM, and can't get the motherboard to fully power on.

Just like with this user here, all that's happening is the fans will move for a fraction of a sec and the Power/Reset buttons are lit up along with a green LED in the lower right corner by the aforementioned buttons.

CPUs and RAM came preinstalled on the motherboard, and were removed to do a proper troubleshoot, with the same outcome. Same thing for trying a different PSU. All power connections are properly seated.

Any recommendations on possible next steps? I've tried looking up the issue, and have reached a dead end.

EDIT: Forgot to ask if the CPU pins on the motherboard are supposed to be at a slight angle, or perfectly upright? I ask, as I noticed the pins in each socket are at a leftwards angle along the bottom and right side and a rightwards angle along the top and left side of the socket.


----------

