It’s been a little over three weeks since Apple unveiled the new iPhone XS, XS Max and the XR. The new flagship line-up is one of Apple’s most important ones to date; this year we’re seeing the company expand last year’s new iPhone X design across all of its new models, meaning big changes for all users. 2018 has been an odd year for the smartphone market as more than ever before we saw the competition try to react and also mimic the iPhone X – the design language especially has been something that we saw replicated across a lot of various designs.

Instead of iterating on the design, Apple has stayed true to its “S” generation release tradition and doubled down on what we saw last year with the iPhone X, all the while expanding the design across new form-factor phones as well. Today we review the first two of this year’s three new models: the iPhone XS, and its bigger brother the iPhone XS Max. The iPhone XR unfortunately releases later on this month – so we'll be taking a look at it separately.

Today we’ll be going into the detail of all aspects of the phone, included a much awaited deep dive of the new A12 SoC. Given Apple's ever-growing focus on the camera capabilities of their phones, I have also prepared an extensive camera comparison for this review – comparing shots across different modes on 14 devices. Hang in tight, this is a long one.

Starting off, let’s go over the specifications of the new iPhone XS and XS Max:

Apple 2018 iPhone XS Specifications
		iPhone XS	iPhone XS Max
SoC		Apple A12 Bionic 2 × Vortex @ 2.5GHz 4 × Tempest @ 1.59GHz
GPU		4-core "G11P" @ >~1.1GHz
DRAM		4GB LPDDR4X
Display		5.8-inch OLED 2436×1125 DCI-P3/True Tone 625 cd/m² brightness 1M:1 contrast ratio 3D Touch	6.5-inch OLED 2688×1242 DCI-P3/True Tone 625 cd/m² brightness 1M:1 contrast ratio 3D Touch
Size	Height	143.6 mm	157.5 mm
	Width	70.9 mm	77.4 mm
	Depth	7.7 mm	7.7 mm
Weight		177 grams	208 grams
Battery Capacity		2658mAh / 10.13Wh	3174mAh / 12.08Wh
Wireless Charging		Qi
Rear Cameras		12 MP 1.4µm pixels, f/1.8, OIS Wide Color Gamut Quad LED True Tone Flash Portrait Mode, Portrait Lighting, Depth Control
		12 MP f/2.4 Telephoto, OIS 2x Optical Zoom Portrait Mode, Portrait Lighting, Depth Control
Front Camera		7MP f/2.2 Wide Gamut, Retina Flash, Portrait Mode, Portrait Lighting, Depth Control
Storage		64 GB 256 GB 512 GB	64 GB 256 GB 512 GB
I/O		Apple Lightning
Wireless (local)		802.11ac Wi-Fi with 2x2 MIMO + Bluetooth 5.0 + NFC
Cellular		Intel XMM7560 Modem UE Category 16 LTE (1Gbps) with 4x4 MIMO and LAA
Splash, Water, Dust Resistance		IP68 up to 2 meters, up to 30 minutes
Dual-SIM		nano-SIM + eSIM nano-SIM + nano-SIM (China model)
Launch Price		64 GB: $999 256 GB: $1149 512 GB: $1349	64 GB: $1099 256 GB: $1249 512 GB: $1449

At the heart of the new iPhones is the brand new Apple A12 SoC. The new chipset is the very first consumer piece of silicon that is being manufactured on TSMC’s new 7nm process. The new process promises greater transistor density and reduced die area of the chip, which gives Apple the ability to cram in more transistors in the same area, ultimately increasing the complexity and capabilities of the new SoC. We’ll go into more detail about the process node in a dedicated section, but least to say, in terms of sheer transistor counts it’s a healthy jump for Apple’s newest design.

The SoC’s CPU complex is now powered by two new “Vortex” CPU cores running at up to 2.5GHz, providing new levels of high performance. Apple claims the new CPUs perform around 15% better than last year’s A11 Monsoon cores – here it seems the company’s marketing was a tad conservative as the real performance figures of the new CPU are significantly higher. Alongside the performant Vortex cores, we see four new smaller efficiency cores named “Tempest”. The new small cores bring some performance improvements, but it’s mostly in terms on power and power efficiency where we see Tempest make some bigger leaps.

The A12’s GPU is the G11P – here Apple has made the biggest performance claims, advertising up to 50% higher figures. We’ll see how the new successor to last year’s A11 GPU in its dedicated section.

On the memory and storage side of things, we now finally see a significant boost in main memory capacity, as both the iPhone XS and XS Max sport 4GB of LPDDR4X RAM, up from 2GB and 3GB in the iPhone 8/X family. In terms of storage capacity, the new models come in 64, 256 and 512GB tiers. Here it’s a tad disappointing to see the base model come in at only 64GB, I think offering 128GB would have resulted in a much more even distribution in the models.

Front and centre of the new iPhones is the new “Super Retina” OLED display. Apple first introduced OLED displays in its lineup in the iPhone X – and the new iPhone XS and XS Max are a continuation of that panel, with an obviously bigger iteration for the Max. The displays are outright fantastic and among the best in the market, offering a fully colour managed wide "Display P3" gamut, very high brightness up to 650 cd/m², and excellent viewing angles. The iPhone XS has the same resolution as last year’s iPhone X, at 2436 x 1125 pixels, while the XS Max maintains the same pixel density by increasing the pixel count to 2688 x 1242.

On the back side, both phones are again a continuation of the iPhone X design. The glass back offers NFC and wireless charging capabilities. Wireless charging has seen an upgrade and claims to be able to charge faster, and also improve on the off-centre and off-axis charging performance.

Apple has worked with Corning to create a new formulation that is said to improve durability and scratch resistance. I’ll leave the testing of this to other people!

The sides of the phones come in highly glossy “surgical grade” steel. While I do appreciate Apple’s intent here, and I understand some people have personal preferences, I’m not a big fan of such finishes as I find them impractical and more slippery than the anodised aluminium of previous generation iPhones. It’s also a hell of a fingerprint magnet.

While the iPhone XS is more or less indistinguishable from the iPhone X – the one visual difference between the new and old is found in the lower left and upper right corners. The left microphone grill has halved its size and is only three holes wide now. Here Apple has introduced two new antenna cut-outs in the corners that serve the two new cellular antennas which enable the iPhone XS’ to achieve 4x4 MIMO.

Indeed in terms of cellular connectivity, the new iPhones boast a significant jump as we’ve seen an upgrade in download speeds to a gigabit for LTE networks. In terms of internals, this was achieved by now exclusively adopting Intel’s new XMM7560 baseband platform. This is Intel’s first chipset to support CDMA and also the first modem to be manufactured on Intel’s own 14nm process.

On the camera side of things we see the same dual 12MP camera module configuration as on the iPhone X – a normal wide angle and a zoom lens. The big difference with the XS is the upgrade in sensor size from an area of 32.8mm² to 40.6mm². Because the lens has remained the same and also offers the same f/1.8 aperture, the increase of the sensor size results in a slightly wider field of view than last year’s models. Light sensitivity has been increased thanks to the bigger sensor, and hence, the bigger pixels, increasing in size from 1.22µm to 1.4µm. The telephoto lens remains largely similar, with a 12MP 2x zoom module.

The iPhone XS Max in contrast to the iPhone XS is just a much bigger device. Apple noted that it’s calling it the “Max” instead of the “Plus” denomination, because it has no added features, and is just a bigger variant of the smaller XS. Here I applaud Apple for not making any feature discrepancies based on the model size – something I really hated in the past with the dual camera being found only on the Plus models.

The iPhone XS Max has a 3174mAh/12.08Wh battery while the smaller XS features a 2658mAh/10.13Wh battery. While Apple calls this the biggest battery ever in an iPhone, and that’s true, Apple is still a tad behind the battery densities that Android manufacturers have now come to commonly use in larger form factor phones.

 

Again, in a comparison between the iPhone X and the new iPhone XS – you’ll be hard pressed to see the difference and you’ll really have to focus to find the new antenna cut-outs to tell them apart.  

 

Meanwhile the iPhone XS Max is largely the same form-factor as the iPhone 8 Plus, although the Max is technically 0.7mm narrower and 0.9mm shorter. For users opting for the larger models, the biggest difference is of course the massive increase in screen size, and I have to say, the XS Max does look very good due to its sheer screen-to-body ratio, which is higher than the smaller XS.

Finally the last big mention before we go deep into our review the price of the new phones. These new models are not successors to the iPhone 8 and 8 Plus – something I imagine the iPhone XR will have much more success in achieving – but rather continuation of the new high price points of the iPhone X. The higher storage capacity price points in particular are exceptional, coming in at $1349 for the XS and $1449 for the XS Max. If the price is worth it for you, is something you best decide along with us as we go deeper into the hardware of the new phones.

The Apple A12 - First Commercial 7nm Silicon

Over the last few years Apple’s silicon design teams have been at the forefront of both architecture design and adopting bleeding-edge manufacturing processes. The Apple A12 is yet another generational jump for the company, as it’s able to claim to be the first commercially available piece of 7nm silicon.

When talking about process nodes, generally speaking the smaller the figure, the smaller the transistor features are. While the actual name of recent nodes have long lost any meaning in correlation to actual physical sizes, they still represent a jump in density, and thus, the ability for vendors to pack in more transistors in the same die space.

We thank TechInsights for publicly sharing their picture of the Apple A12, and we’ve followed up with posting a quick first analysis reaction to the die shot:

AnandTech modified TechInsights Apple A12 Die Shot

Going over it again here for this article, I’ve put down my own custom labelling and interpretation of the die shot. The new A12 largely follows Apple’s SoC layout structure (90° rotated to most past die shots).

On the right side we see the GPU complex with the four GPU cores and shared logic in the middle. The CPU complex is found at the bottom, with the two Vortex big CPU cores on the centre-left, divided by the big L2 cache, right next to the four small Tempest CPU cores and their own L2 cache.

The four big chunks of SRAM in the middle blocks are part of the system cache – this is a SoC-wide cache layer in between the memory controllers and the internal system interconnect & block memory subsystems. Here Apple uses this block as an energy saving feature: Because memory transactions to DRAM are quite expensive in terms of energy usage, being able to cache things on-chip will save a great amount of power, with the added benefit of possible performance increases due to the locality of the data.

The Apple A12’s system cache has, to date, seen the biggest change since its introduction in the Apple A7. The big change in layout also points out to a large change in the functionality of the block, as now we clearly see a separation of the block into what’s apparent to be four slices. On previous Apple SoCs such as in the A11 or A10, the system cache looked more like a single block of logic, with what appeared to be two slices. A doubling of the slices could possibly point out to a very large change in the memory performance of this block – something I very much believe to be the case following more analysis in the coming pages.

Finally, the last big introduction in the A12 is a major revamp of the neural accelerator IP. Here Apple claims to have moved from a dual-core design found in the A11, to a new 8-core design. Last year’s design was rumoured to be a CEVA IP – although we’ve never have full confirmation on that as Apple doesn’t want it to be known. 

During the keynote presentation it’s important to note that Apple never mentioned that this is an in-house design, something the marketing materials was always eager to point out for other IP blocks of the SoCs.

Edit: I've been notified that Apple does have the "Apple-designed" mention on the A12 webpage, meaning this does indeed mean it's an in-house IP.

The A12 being an 8-core design would point out to a 4x increase in performance – but the actual increase is near 8x, increasing from 600GigaOPs in the A11 to 5TeraOPs in the A12. In the die shot we see the 8 MAC engines surrounding a big central cache, with possible shared logic at the top that would be responsible for fixed function and fully connected layer processing.

Die Block Comparison (mm²)
SoC Process Node	Apple A12 TSMC N7	Apple A11 TSMC 10FF
Total Die	83.27	87.66
Big Core	2.07	2.68
Small Core	0.43	0.53
CPU Complex (incl. cores)	11.90	14.48
GPU Total	14.88	15.28
GPU Core	3.23	4.43
NPU	5.79	1.83

Looking over the different block size changes from the A11 to the A12, we see the benefits of TSMC’s newer 7nm manufacturing node. It’s to be noted that nearly all IP blocks have undergone changes, so it’s not really a valid apples-to-apples comparison to determine just how much density has improved with the new manufacturing node. Still, taking a single GPU core as a possible candidate (as we largely see the same structure), we see a 37% decrease in size compared to the A11. It’s quite obvious that the new node has enabled Apple to add an additional GPU core even though in absolute terms, the GPU is still smaller in the A12.

A bigger CPU and a massive cache hierarchy

Credit: TechInsights Apple A12 Die Shot, ChipRebel Apple A11 Die Shot

Moving on to the CPU complex, and in particular the new big CPU core, we now see what is possibly Apple’s biggest change in its CPU layout in several generations. In particular, we see a doubling of the L1 data caches in the new Vortex CPU, increasing from 64KB to 128KB in the new core. In the front-end, we also saw a doubling of the SRAM blocks that I attribute to the L1 instruction caches – which I now believe to have also seen doubling to 128KB. It’s funny that even after several years, we still haven’t really figured out what the A10 had introduced into the front-end block: here we saw a new very large block of cache whose function largely remains unclear.

A big question over the years has been what Apple’s cache hierarchy actually looks like. Looking at the memory latency behaviour at different test depths, we see the different jumps at different test depths. I didn’t annotate the latency values as we’ll revisit them later in a non-logarithmic version of this graph.

On the big core side, we clearly see the L1$ jump from 64KB to 128KB, and I think there’s no doubt about the increases here. Moving into the L2 cache however, we’re seeing some odd behaviour in terms of the latency. It’s clear that around the 3MB range that there’s an increasing jump in latency, up until around 6MB. It’s to be noted that this behaviour of a slower second 3MB only happens when accessing in a fully random patterns, in smaller access windows the latency is flat up until 6MB.

Not dwelling more into this for now, we move into the area beyond 6MB that is served by the system cache. It’s hard to make it out at first because there’s an offset caused by overall lower latency, but in general the latency curve goes around 4MB further before we reach mostly DRAM latencies. This corresponds to what we actually see on the die: the new system cache not only has seen a doubling of its slices, but has also outright doubled in capacity from 4 to 8MB.

Moving onto an analysis of the little cores, things become a bit more complex. At first glance you would believe the A11’s small cores L2 was limited to 512KB and that the A12 goes up to 1.5MB, however what I think is going on is that we’re being tricked by the power management policy of the cache. Looking at the A11 Mistral core latency, we see some obvious jumps at 768KB and 1MB. A similar jump can be seen in the A12 cores at 2MB.

It’s at this point, where it’s best to go back to the die shots and do some pixel counting, and we come up with the following table:

Measured and Estimated Cache Sizes
SoC	Apple A12	Apple A11
Big L1$	128KB	64KB
Big L2$	128 instances 6MB per core/thread 8MB total at 64KB/inst	128 instances 6MB per core/thread 8MB total at 64KB/inst
Small L1$	32KB	32KB
Small L2$	32 instances 1.5MB per core/thread 2MB total at 64KB/inst	16 + 2(?) instances 512KB per core/thread 1MB total at 64KB/inst
System Cache	4x 16 instances (double size) 8MB at 128KB/inst	2x 32 instances 4MB at 64KB/inst

The big core L2’s haven’t seen any structural changes between the A11 and A12 – both have 128 instances of SRAM macros, separated into two chunks. The question here remains that if the L2 is indeed just 6MB, then that would mean 48KB per SRAM block.

When looking at the small cores, we see that they use identical SRAM macros. The A12’s small core L2 has doubled from 16 to 32 instances, meaning there must have been a doubling. However as we see that the measured latency depth of the L2 has at least tripled, something else must be going on. What we’re measuring by no means has to represent what’s in the hardware, and indeed we can confirm this by running the latency test in a more special manner that makes the power management think it’s just some minor workload. On the A12, the Tempest cores then appear to have only 512KB available to them.

The conclusion is here is that Apple is employing partial cache power-down in what seems to be per-bank granularity. On the A12 each small core L2 bank is 512KB, and on the A11 it’s 256KB. Also, this more strongly leads me to believe there’s 2MB on the A12 and 1MB on the A11, it’s just that the test probably doesn’t fulfill the policy requirements to access the full cache.

In turn, because this would confirm 64KB per SRAM instance, we can go back and make some hypotheses about the big core L2’s. Again, looking at it, one would believe it stops at 6MB, but looking closer, especially on the A12, there is a change of behaviour at 8MB. Again it’s likely that the cores have 8MB of physical cache, and there’s an obvious change in access behaviour once we’re nearing a full cache.

The take-away here is that Apple’s caches are just immense, and the A12 further expands in that regard by doubling the system cache size. In practice, we have around 16MB of useable cache hierarchy on the part of the big CPU cores – a massive amount that just puts the memory and cache subsystems of competing SoCs to shame.

An evolutionary GPU

On the GPU side of things, we had some big expectations from the A12, not merely in terms of performance, but also in terms of architecture. Last year there had been a press release from Imagination stating that Apple had informed them that the company planned to no longer use its IP in new products in 15 to 24 months’ time. This would eventually lead to and crash of the stock price and subsequent sale of the company to an equity firm.  

So while Apple did declare the A11 GPU as an in-house design, it still very much looked like an Imagination derived design, as its block design was very similar to prior Rogue generations – with the big difference being that what was now called a core is the larger structure of what was previously two cores. The fact that it’s still a TBDR (Tile-Based Deferred Renderer), which IMG holds many patents on, but not least the fact that Apple still very much exposes and supports PVRTC (PowerVR Texture Compression), a proprietary format, means that the GPU still likely linked to IMG’s IP. Here it’s likely that we’re still looking at an architectural license design rather than what we would usually call a “clean sheet” design.

Credit: TechInsights Apple A12 Die Shot, ChipRebel Apple A11 Die Shot

Moving onto the A12 GPU – model named as the G11P, we see some very obvious similarities to last year’s A11 GPU. The individual function blocks seem to be largely located the same and constructed in a similar fashion.

What I think Apple’s biggest advancements in the A12 GPU is the fact that it now supports memory compression. I was very surprised to hear this during the announcement because at the same time it means two things: Prior Apple SoCs and their GPUs apparently didn’t have memory compression, and that now this alone should amount for a very significant boost in performance of the new GPU.

By memory compression, in particular we mean transparent framebuffer compression from the GPU to main memory. In the desktop space, vendors like Nvidia and AMD have had this for many years now, and it enabled the advancement of GPU performance even in the face of non-advancing memory-bandwidth increases. Smartphone GPUs also require memory compression, not only because of limited bandwidth on mobile SoCs, but most importantly because of the reduced power consumption that is associated with high bandwidth requirements. Arm’s AFBC has been the most publicly talked about mechanism in the mobile space, but other vendors such as Qualcomm or even Imagination have their own implementations.

Apple seems to be very late to the party in only introducing this with the A12 – however it also means the A12 will benefit from an unusually large generational boost in efficiency and performance, which makes a lot of sense given Apple’s proclaimed large increases for the new GPU.

The A12 Vortex CPU µarch

When talking about the Vortex microarchitecture, we first need to talk about exactly what kind of frequencies we’re seeing on Apple’s new SoC. Over the last few generations Apple has been steadily raising frequencies of its big cores, all while also raising the microarchitecture’s IPC. I did a quick test of the frequency behaviour of the A12 versus the A11, and came up with the following table:

Maximum Frequency vs Loaded Threads Per-Core Maximum MHz
Apple A11	1	2	3	4	5	6
Big 1	2380	2325	2083	2083	2083	2083
Big 2		2325	2083	2083	2083	2083
Little 1			1694	1587	1587	1587
Little 2				1587	1587	1587
Little 3					1587	1587
Little 4						1587
Apple A12	1	2	3	4	5	6
Big 1	2500	2380	2380	2380	2380	2380
Big 2		2380	2380	2380	2380	2380
Little 1			1587	1562	1562	1538
Little 2				1562	1562	1538
Little 3					1562	1538
Little 4						1538

Both the A11 and A12’s maximum frequency is actually a single-thread boost clock – 2380MHz for the A11’s Monsoon cores and 2500MHz for the new Vortex cores in the A12. This is just a 5% boost in frequency in ST applications. When adding a second big thread, both the A11 and A12 clock down to respectively 2325 and 2380MHz. It’s when we are also concurrently running threads onto the small cores that things between the two SoCs diverge: while the A11 further clocks down to 2083MHz, the A12 retains the same 2380 until it hits thermal limits and eventually throttles down.

On the small core side of things, the new Tempest cores are actually clocked more conservatively compared to the Mistral predecessors. When the system just had one small core running on the A11, this would boost up to 1694MHz. This behaviour is now gone on the A12, and the clock maximum clock is 1587MHz. The frequency further slightly reduces to down to 1538MHz when there’s four small cores fully loaded.

Much improved memory latency

As mentioned in the previous page, it’s evident that Apple has put a significant amount of work into the cache hierarchy as well as memory subsystem of the A12. Going back to a linear latency graph, we see the following behaviours for full random latencies, for both big and small cores:

The Vortex cores have only a 5% boost in frequency over the Monsoon cores, yet the absolute L2 memory latency has improved by 29% from ~11.5ns down to ~8.8ns. Meaning the new Vortex cores’ L2 cache now completes its operations in a significantly fewer number of cycles. On the Tempest side, the L2 cycle latency seems to have remained the same, but again there’s been a large change in terms of the L2 partitioning and power management, allowing access to a larger chunk of the physical L2.

I only had the test depth test up until 64MB and it’s evident that the latency curves don’t flatten out yet in this data set, but it’s visible that latency to DRAM has seen some improvements. The larger difference of the DRAM access of the Tempest cores could be explained by a raising of the maximum memory controller DVFS frequency when just small cores are active – their performance will look better when there’s also a big thread on the big cores running.

The system cache of the A12 has seen some dramatic changes in its behaviour. While bandwidth is this part of the cache hierarchy has seen a reduction compared to the A11, the latency has been much improved. One significant effect here which can be either attributed to the L2 prefetcher, or what I also see a possibility, prefetchers on the system cache side: The latency performance as well as the amount of streaming prefetchers has gone up.

Instruction throughput and latency

Backend Execution Throughput and Latency
	Cortex-A75		Cortex-A76		Exynos-M3		Monsoon | Vortex
	Exec	Lat	Exec	Lat	Exec	Lat	Exec	Lat
Integer Arithmetic ADD	2	1	3	1	4	1	6	1
Integer Multiply 32b MUL	1	3	1	2	2	3	2	4
Integer Multiply 64b MUL	1	3	1	2	1 (2x 0.5)	4	2	4
Integer Division 32b SDIV	0.25	12	0.2	< 12	1/12 - 1	< 12	0.2	10 | 8
Integer Division 64b SDIV	0.25	12	0.2	< 12	1/21 - 1	< 21	0.2	10 | 8
Move MOV	2	1	3	1	3	1	3	1
Shift ops LSL	2	1	3	1	3	1	6	1
Load instructions	2	4	2	4	2	4	2
Store instructions	2	1	2	1	1	1	2
FP Arithmetic FADD	2	3	2	2	3	2	3	3
FP Multiply FMUL	2	3	2	3	3	4	3	4
Multiply Accumulate MLA	2	5	2	4	3	4	3	4
FP Division (S-form)	0.2-0.33	6-10	0.66	7	>0.16	12	0.5 | 1	10 | 8
FP Load	2	5	2	5	2	5
FP Store	2	1-N	2	2	2	1
Vector Arithmetic	2	3	2	2	3	1	3	2
Vector Multiply	1	4	1	4	1	3	3	3
Vector Multiply Accumulate	1	4	1	4	1	3	3	3
Vector FP Arithmetic	2	3	2	2	3	2	3	3
Vector FP Multiply	2	3	2	3	1	3	3	4
Vector Chained MAC (VMLA)	2	6	2	5	3	5	3	3
Vector FP Fused MAC (VFMA)	2	5	2	4	3	4	3	3

To compare the backend characteristics of Vortex, we’ve tested the instruction throughput. The backend performance is determined by the amount of execution units and the latency is dictated by the quality of their design.

The Vortex core looks pretty much the same as the predecessor Monsoon (A11) – with the exception that we’re seemingly looking at new division units, as the execution latency has seen a shaving of 2 cycles both on the integer and FP side. On the FP side the division throughput has seen a doubling.

Monsoon (A11) was a major microarchitectural update in terms of the mid-core and backend. It’s there that Apple had shifted the microarchitecture in Hurricane (A10) from a 6-wide decode from  to a 7-wide decode. The most significant change in the backend here was the addition of two integer ALU units, upping them from 4 to 6 units.

Monsoon (A11) and Vortex (A12) are extremely wide machines – with 6 integer execution pipelines among which two are complex units, two load/store units, two branch ports, and three FP/vector pipelines this gives an estimated 13 execution ports, far wider than Arm’s upcoming Cortex A76 and also wider than Samsung’s M3. In fact, assuming we're not looking at an atypical shared port situation, Apple’s microarchitecture seems to far surpass anything else in terms of width, including desktop CPUs.

SPEC2006 Performance: Reaching Desktop Levels

It’s been a while now since we attempted SPEC on an iOS device – for various reasons we weren’t able to continue with that over the last few years. I know a lot of people were looking forward to us picking back up from where we left, and I’m happy to share that I’ve spent some time getting a full SPEC2006 harness back to work.

SPEC2006 is an important industry standard benchmark and differentiates itself from other workloads in that the datasets that it works on are significantly larger and more complex. While GeekBench 4 has established itself as a popular benchmark in the industry – and I do praise on the efforts on having a full cross-platform benchmark – one does have to take into account that it’s still relatively on the light side in terms of program sizes and the data sizes of its workloads. As such, SPEC2006 is much better as a representative benchmark that fully exhibits more details of a given microarchitecture, especially in regards to the memory subsystem performance.

The following SPEC figures are declared as estimates, as they were not submitted and officially validated by SPEC. The benchmark libraries were compiled with the following settings:

• Android: Toolchain: NDK r16 LLVM compiler, Flags: -Ofast, -mcpu=cortex-A53
• iOS: Toolchain: Xcode 10, Flags: -Ofast

On iOS, 429.mcf was a problem case as the kernel memory allocator generally refuses to allocate the single large 1.8GB chunk that the program requires (even on the new 4GB iPhones). I’ve modified the benchmark to use only half the amount of arcs, thus roughly reducing the memory footprint to ~1GB. The reduction in runtime has been measured on several platforms and I’ve applied a similar scaling factor to the iOS score – which I estimate to being +-5% accurate. The remaining workloads were manually verified and validated for correct execution.

The performance measurement was run in a synthetic environment (read: bench fan cooling the phones) where we assured thermals wouldn’t be an issue for the 1-2 hours it takes to complete a full suite run.

In terms of data presentation, I’m following of earlier articles this year such as the Snapdragon 845 and Exynos 9810 evaluation in our Galaxy S9 review.

When measuring performance and efficiency, it’s important to take three metrics into account: Evidently, the performance and runtime of a benchmark, which in the graphs below is represented on the right axis, growing from the right. Here the bigger the figures, the more performant a SoC/CPU has benchmarked. The labels represent the SPECspeed scores.

On the left axis, the bars are representing the energy usage for the given workload. The bars grow from the left, and a longer bar means more energy used by the platform. A platform is more energy efficient when the bars are shorter, meaning less energy used. The labels showcase the average power used in Watts, which is still an important secondary metric to take into account in thermally constrained devices, as well as the total energy used in Joules, which is the primary efficiency metric.

The data is ordered as in the legend, and colour coded by different SoC vendor as well as shaded by the different generations. I’ve kept the data to the Apple A12, A11, Exynos 9810 (at 2.7 and 2.3GHz), Exynos 8895, Snapdragon 845 and Snapdragon 835. This gives us an overview of all relevant CPU microarchitectures over the last two years.

Starting off with the SPECint2006 workloads:

The A12 clocks in at 5% higher than the A11 in most workloads, however we have to keep in mind we can’t really lock the frequencies on iOS devices, so this is just an assumption of the runtime clocks during the benchmarks. In SPECint2006, the A12 performed an average of 24% better than the A11.

The smallest increases are seen in 456.hmmer and 464.h264ref – both of these tests are the two most execution bottlenecked tests in the suite. As the A12 seemingly did not really have any major changes in this regard, the small increase can be mainly attributed to the higher frequency as well as the improvements in the cache hierarchy.

The improvements in 445.gobmk are quite large at 27% - the characteristics of the workload here are bottlenecks in the store address events as well as branch mispredictions. I did measure that the A12 had some major change in the way stores across cache lines were handled, as I’m not seeing significant changes in the branch predictor accuracy.

403.gcc partly, and most valid for 429.mcf, 471.omnetpp, 473.Astar and 483.xalancbmk are sensible to the memory subsystem and this is where the A12 just has astounding performance gains from 30 to 42%. It’s clear that the new cache hierarchy and memory subsystem has greatly paid off here as Apple was able to pull off one of the most major performance jumps in recent generations.

When looking at power efficiency – overall the A12 has improved by 12% - but we have to remember that we’re talking about 12% less energy at peak performance. The A12 showcasing 24% better performance means were comparing two very different points at the performance/power curve of the two SoCs.

In the benchmarks where the performance gains were the largest – the aforementioned memory limited workloads – we saw power consumption rise quite significantly. So even though 7nm promised power gains, Apple's opted to spend more energy than what the new process node has saved, so average power across the totality of SPECint2006 did go up from ~3.36W on the A11 to 3.64W on the A12.

Moving on to SPECfp2006, we are looking at the C and C++ benchmarks, as we have no Fortran compiler in XCode, and it is incredibly complicated to get one working for Android as it’s not part of the NDK, which has a deprecated version of GCC.

SPECfp2006 has a lot more tests that are very memory intensive – out of the 7 tests, only 444.namd, 447.dealII, and 453.povray don’t see major performance regressions if the memory subsystem isn’t up to par.

Of course this majorly favours the A12, as the average gain for SPECfp is 28%. 433.milc here absolutely stands out with a massive 75% gain in performance. The benchmark is characterised by being instruction store limited – again part of the Vortex µarch that I saw a great improvement in. The same analysis applies to 450.soplex – a combination of the superior cache hierarchy and memory store performance greatly improves the perf by 42%.

470.lbm is an interesting workload for the Apple CPUs as they showcase multi-factor performance advantages over competing Arm and Samsung cores. Qualcomm’s Snapdragon 820 Kryo CPU oddly enough still outperforms the recent Android SoCs. 470.lbm is characterised by extremely large loops in the hottest piece of code. Microarchitectures can optimise such workloads by having (larger) instruction loop buffers, where on a loop iteration the core would bypass the decode stages and fetch the instructions from the buffer. It seems that Apple’s microarchitecture has some kind of such a mechanism. The other explanation is also the vector execution performance of the Apple cores – lbm’s hot loop makes heavy use of SIMD, and Apple’s 3x execution throughput advantage is also likely a heavy contributor to the performance.

Similar to SPECint, the SPECfp workload which saw the biggest performance jumps also saw an increase in their power consumption. 433.milc saw an increase from 2.7W to 4.2W, again with a 75% performance increase.

Overall the power consumption has seen a jump from 3.65W up to 4.27W. The overall energy efficiency has increased in all tests but 482.sphinx3, where the power increase hit the maximum across all SPEC workloads for the A12 at 5.35W. The total energy used for SPECfp2006 for the A12 is 10% lower than the A11.

I didn’t have time to go back and measure the power for the A10 and A9, but generally they’re in line around 3W for SPEC. I did run the performance benchmarks, and here’s an aggregate performance overview of the A9 through to the A12 along with the most recent Android SoCs, for those who are looking into comparing past Apple generations.

Overall the new A12 Vortex cores and the architectural improvements on the SoC’s memory subsystem give Apple’s new piece of silicon a much higher performance advantage than Apple’s marketing materials promote. The contrast to the best Android SoCs have to offer is extremely stark – both in terms of performance as well as in power efficiency. Apple’s SoCs have better energy efficiency than all recent Android SoCs while having a nearly 2x performance advantage. I wouldn’t be surprised that if we were to normalise for energy used, Apple would have a 3x performance lead.

This also gives us a great piece of context for Samsung’s M3 core, which was released this year: the argument that higher power consumption brings higher performance only makes sense when the total energy is kept within check. Here the Exynos 9810 uses twice the energy over last year’s A11 – at a 55% performance deficit.

Meanwhile Arm’s Cortex A76 is scheduled to arrive inside the Kirin 980 as part of the Huawei Mate 20 in just a couple of weeks – and I’ll be making sure we’re giving the new flagship a proper examination and placing among current SoCs in our performance and efficiency graph.

The A12 Tempest CPU µarch: A Fierce Small Core

Apple had first introduced a “small” CPU core alongside the Twister cores in the A10 SoC, powering the iPhone 7 generation. We’ve never really had the opportunity to dissect these cores, and over the years there was a bit of mystery around them as to what they’re capable of.

Apple’s introduction of a heterogeneous CPU topology in one sense was one of the biggest validations for Arm designs. Having separate low(er)-power CPUs on a SoC is a simple matter of physics: It’s just not possible to have bigger microarchitectures scale down power as efficiently as if you would just use a separate smaller block. Even in a mythical perfectly clock-gated microarchitecture, you would not be able to combat the static leakage present in bigger CPU cores, and thus this would come with the negative consequence of being part of the everyday power consumption on a device, even for small workloads. Power gating the big CPU cores, and instead shifting to much smaller CPU in contrast, helps alleviate static leakage, as well as (if designed as such) improving the dynamic leakage power efficiency.

The Tempest cores in the A12 are now the third iteration of this “small” microarchitecture, and since the A11 they are now fully heterogeneous and work independently of the big cores. But the question is, is this actually the third iteration, or did Apple do something more interesting?

The Tempest core is a 3-wide out-of-order microarchitecture: Already out of the gate this means it has very little to do with Arm’s own “little” cores, such as the A53 and A55, as these are simpler in-order designs.

The Tempest core’s execution pipelines are also relatively few: There are just two main pipelines that are capable of simple ALU operations; meanwhile one of them also does integer and FP multiplications, and the other is able to do FP additions. Essentially we just have two primary execution ports to each of the more complex pipelines behind them. Meanwhile in addition to the two main pipelines, there’s also a dedicated combined load/store port.

Now what is very interesting here is that this essentially looks identical to Apple’s Swift microarchitecture from Apple's A6 SoC. It’s not very hard to imagine that Apple would have recycled this design, ported it to 64-bit, and they now use it as a lean out-of-order machine serving as the lower power CPU core. If this is indeed Swift derived, then on top of the three execution ports described above, we should also find a dedicated port for integer and fp divisions, such as not to block the main pipelines whenever such an instruction is fed.

The Tempest cores clock up to a maximum of 1587MHz and are served by 32KB instruction and data caches, as well as an increased shared 2MB L2 cache that uses power management to partially power down SRAM banks.

In terms of power efficiency, the Tempest cores were essentially my prime candidate to try to get to some sort of apples-to-apples comparison between the A11 and A12 for power efficiency. I haven’t seen major differences in the cores besides the bigger L2, and Apple has also kept the frequencies similar. Unfortunately, "similar" isn't identical in this case; because the small cores on the A11 can boost up to 1694MHz when there’s only one thread active on them, I had no really good way to also measure performance at iso-frequency.

I did run SPEC at an equal 1587MHz frequency by simply having a second dummy thread spinning on another core while the main workloads were benchmarking. And I did try to get some power figures through this method by regression testing the impact of the dummy thread. However the power was near identical to the figures I measured at 1694MHz. As a result I dropped the idea, and we'll just have to just keep in mind that the A11’s Mistal cores were running 6.7% faster in the following benchmarks:

Much like on the Vortex big cores, the biggest improvements for the new Tempest cores are found in the memory-sensitive benchmarks. The benchmarks in which Tempest loses to Mistral are mainly execution bound, and because of the frequency disadvantage, there’s no surprise that the A12 lost in this particular single-threaded small core scenario.

Overall, besides the memory improvements, the new Tempest cores looks very similar in performance to last year’s Mistral cores. This is great as we can also investigate the power efficiency, and maybe learn something more concrete about the advantages of TSMC's 7nm manufacturing process.

Unfortunately, the energy efficiency improvements are somewhat inconclusive, and more so maybe disappointing. Looking at the SPECint2006 workloads overall, the Tempest-powered A12 was 35% more energy efficient than the Mistral-powered A11. Because the Mistral cores were running at a higher frequency in this test, the actual efficiency gains for A12 would likely be even less at an ISO-frequency level. Granted, we’re still looking at a general ISO-performance comparison here, as the memory improvements in A12 were able to push the Tempest cores to an integer suite score nearly identical to the higher-clocked Mistral cores.

In the overall FP benchmarks, Tempest was only 17% more efficient, even though it did perform better than the A11’s Mistral cores.

Putting the A11 and A12 small cores in comparison with their big brothers as well as the competition from Arm, there’s not much surprise in terms of the results. Compared to the big Apple cores, the small cores only offer a third to a fourth of the performance, but they also use less than half the energy.

What did surprise me a lot was seeing just how well Apple’s small cores compare to Arm’s Cortex-A73 under SPECint. Here Apple’s small cores almost match the performance of Arm’s high-performance cores from ust 2 years ago. In SPEC's integer workloads, A12 Tempest is nearly equivalent to a 2.1GHz A73.

However in the SPECfp workloads, the small cores aren’t competitive. Not having dedicated floating-point execution resources puts the cores at a disadvantage, though they still offer great energy efficiency.

Apple’s small cores in general are a lot more performant that one would think. I’ve gathered some incomplete SPEC numbers on Arm’s A55 (it takes ages!) and in general the performance difference here is 2-3x depending on the benchmark. In recent years I’ve felt that Arm’s little core performance range has become insufficient in many workloads, and this may also be why we’re going to see a lot more three-tiered SoCs (such as the Kirin 980) in the coming future. As it stands, the gap between the maximum performance of the little cores and the most efficient low performance point of the big continues to grow into one direction. All of which makes me wonder whether it’s still worth it to stay with an in-order microarchitecture for Arm's efficiency cores.

Neural Network Inferencing Performance on the A12

Another big, mysterious aspect of the new A12 was the SoC's new neural engine, which Apple advertises as designed in-house.  As you may have noticed in the die shot, it’s a quite big silicon block, very much equaling the two big Vortex CPU cores in size.

To my surprise, I found out that Master Lu’s AImark benchmark also supports iOS, and better still it uses Apple's CoreML framework to accelerate the same inference models as on Android. I ran the benchmark on the latest iPhone generations, as well as a few key Android devices.

Overall, Apple’s 8x performance claims weren’t quite confirmed in this particular test suite, but we see solid improvements of 4-6.5x. There’s one catch here in regards to the older iPhones: as you can see in the results, the A11-based iPhone X performs quite similarly to previous generation phones. What’s happening here is that Apple’s executing CoreML on the GPU. It seems to me that the NPU in the A11 might have never been exposed publicly via APIs.

The Huawei P20 Pro’s Kirin 970 falls roughly 2.5x behind the new A12 – which coincidentally exactly matches the advertised 2TOPs vs 5TOPs throughout capabilities of both SoC’s respective NPUs. Here the new Kirin 980 should be able to significantly close the gap.

Qualcomm’s Snapdragon 845 also performs very well, trading blows with the Kirin 970. AImark uses the SNPE framework for inference acceleration, as it doesn’t support the NNAPI as of yet. The Pixel 2 and Note9 offered terrible results here as they both had to fall back to CPU accelerated libraries.

In terms of power, I’m not too comfortable publishing power on the A12 because of how the workload was visibly transactional: The actual inferencing workload bumped up power consumption up to 5.5W, with lower gaps in-between. Without actually knowing what is happening in-between the bursts of activity, the average power figures for the whole test run can vary greatly. Nevertheless, the fact that Apple’s willing to go up to 5.5W means that they’re very much pushing the power envelope here and going for the highest burst performance. The GPU-accelerated iPhone’s power peaked in the 2.3W to 5W range depending on the inference model.

System Performance

While synthetic test performance is one thing, and hopefully we’ve covered that well with SPEC, interactive performance in real use-cases behaves differently, and here software can play a major role in terms of the perceived performance.

I will openly admit that our iOS system performance suite looks extremely meager: we are only really left with our web browser tests, as iOS is quite lacking in meaningful alternatives such as to PCMark on the Android side.

Speedometer 2.0 is the most up-to-date industry standard JavaScript benchmark which tests the most common and modern JS framework performance.

The A12 sports a massive jump of 31% over the A11, again pointing out that Apple’s advertised performance figures are quite underselling the new chipset.

We’re also seeing a small boost from iOS 12 on the previous generation devices. Here the boost comes not only thanks to an a change in how iOS’s scheduler handles load, but also thanks to further improvements in the ever evolving JS engine that Apple uses.

WebXPRT 3 is also a browser test, however its workloads are more wide-spread and varied, containing also a lot of processing tests. Here the iPhone XS showcases a smaller 11% advantage over the iPhone X.

Former devices here also see a healthy boost in performance, with the iPhone X ticking up from 134 to 147 points, or 10%. The iPhone 7’s A10 sees a larger boost of 33%, something we’ll get into more detail in a little bit.

iOS12 Scheduler Load Ramp Analyzed

Apple promised a significant performance improvement in iOS12, thanks to the way their new scheduler is accounting for the loads from individual tasks. The operating system’s kernel scheduler tracks execution time of threads, and aggregates this into an utilisation metric which is then used by for example the DVFS mechanism. The algorithm which decides on how this load is accounted over time is generally simple a software decision – and it can be tweaked and engineered to whatever a vendor sees fit.

Because iOS’s kernel is closed source, we’re can’t really see what the changes are, however we can measure their effects. A relatively simple way to do this is to track frequency over time in a workload from idle, to full performance. I did this on a set of iPhones ranging from the 6 to the X (and XS), before and after the iOS12 system update.

Starting off with the iPhone 6 with the A8 chipset, I had some odd results on iOS11 as the scaling behaviour from idle to full performance was quite unusual. I repeated this a few times yet it still came up with the same results. The A8’s CPU’s idled at 400MHz, and remained here for 110ms until it jumped to 600MHz and then again 10ms later went on to the full 1400MHz of the cores.

iOS12 showcased a more step-wise behaviour, scaling up earlier and also reaching full performance after 90ms.

The iPhone 6S had a significantly different scaling behaviour on iOS11, and the A9 chip’s DVFS was insanely slow. Here it took a total of 435ms for the CPU to reach its maximum frequency. With the iOS12 update, this time has been massively slashed down to 80ms, giving a great boost to performance in shorter interactive workloads.

I was quite astonished to see just how slow the scheduler was before – this is currently the very same issue that is handicapping Samsung’s Exynos chipsets and maybe other Android SoCs who don’t optimise their schedulers. While the hardware performance might be there, it just doesn’t manifest itself in short interactive workloads because the scheduler load tracking algorithm is just too slow.

The A10 had similar bad characteristics as the A9, with time to full performance well exceeding 400ms. In iOS12, the iPhone 7 slashes this roughly in half, to around 210ms. It’s odd to see the A10 being more conservative in this regard compared to the A9 – but this might have something to do with the little cores.

In this graph, it’s also notable to see the frequency of the small cores Zephyr cores – they start at 400MHz and peak at 1100MHz. The frequency in the graph goes down back to 758MHz because at this point there was a core switch over to the big cores, which continue their frequency ramp up until maximum performance.

On the Apple A11 – I didn’t see any major changes, and indeed any differences could just be random noise between measuring on the different firmwares. Both in iOS11 and iOS12, the A11 scales to full frequency in about 105ms. Please note the x-axis in this graph is a lot shorter than previous graphs.

Finally on the iPhone XS’s A12 chipset, we can’t measure any pre- and post- update as the phone comes with iOS12 out of the box. Here again we see that it reaches full performance after 108ms, and we see the transition of the tread from the Tempest cores over to the Vortex cores.

Overall, I hope this is the best and clear visual representation of the performance differences that iOS12 brings to older devices.

In terms of the iPhone XS – I haven’t had any issues at all with performance of the phone and it was fast. I have to admit I’m still a daily Android user, and I use my phones with animations completely turned off as I find they get in the way of the speed of a device. There’s no way to completely turn animation off in iOS, and while this is just my subjective personal opinion, I found they are quite hampering the true performance of the phone. In workloads that are not interactive, the iPhone XS just blazed through them without any issue or concern.

GPU Performance

The performance improvement of the A12 GPU was one of the biggest highlights of the keynote presentation, promising up to 50% higher performance versus the A11 GPU. Apple has achieved this by "simply" adding in a fourth GPU core, up from three on the A11, and by introducing memory compression on the GPU. The memory compression is what I think the most contributing factor to the increased microarchitectural performance of the GPU, as it really is a huge one-time shift, which admittedly, took Apple a long time to make.

One thing that I’d like to mention before going into the benchmarks, is that peak performance and peak power consumption of the latest Apple GPUs is a problem. We’ve seen Apple transition from promoting its sustained performance over time, to actually being one of the worst “offenders” in terms of actual performance degradation from the peak capabilities of the SoC. There’s reasons to this, but I’ll be addressing them shortly.

In the 3DMark Physics test, which is mostly a CPU-bound test that also stresses the overall platform power limits while the GPU is also doing work, we see the iPhone XS and the A12 achieve some great gains over last year’s iPhone. This had been a test that in the past had been particularly problematic for Apple CPUs, however it seems that this microarchitectural hiccup was solved in the A11 and the Monsoon cores. The Vortex cores along with the generally improved power efficiency of the SoC further raises the performance, finally matching the Arm’s cores in this particular test.

In the Graphics part of the 3DMark test, the iPhone XS showcases 41% better sustained performance over last year’s iPhone X. In this particular test, the OnePlus 6’s more generous thermals still allow the Snapdragon 845 to outperform the new chip.

In terms of peak performance, I encountered some great issues in 3DMark: I was completely unable to complete a single run on either the iPhone XS or XS Max while the devices were cool. If the device is cool enough, the GPU will boost to such high performance states that it will actually crash. I was consistently able to reproduce this over and over again. I attempted to measure power during this test, and the platform had instantaneous average power of 7-8 watts, figures above this which I suspect weren’t recorded by my measurement methodology. For the GPU to crash, it means that the power delivery is failing to deliver the necessary transient currents during operation and we’ll see a voltage dip that corrupts the GPU.

When iterating the test several times over a few attempts, in order to heat up the SoC until it decides to start off with a lower GPU frequency, it will successfully complete the test.

GFXBench

Kishonti most recently released the new GFXBench 5 Aztec Ruins test, which brings a newer, more modern, and complex workload to our test suite. In an ideal world we would be testing real games, however this is an incredible headache on mobile devices as there are essentially no games with built-in benchmarking modes. There are some tools to gather fps values, but the biggest concern here is repeatability of the workload when one manually plays the game – also a huge concern for many of the online games of today.

GFXBench Sub-Tests
AnandTech		Aztec High	Aztec Normal	Manhattan 3.1	T-Rex
Scene length		64.3s	64.3s	62s	56s
Resolution		2560 x 1440	1920 x 1080	1920 x 1080	1920 x 1080
Compute Shaded Pixels		~1.5% of work	~1.5% of work	~3% of work	~2.4% of work
Total Shaded Pixels		~5.80M / frame ~161% of scene	~2.64M / frame ~127% of scene	~1.90M / frame ~92% of scene	~0.65M / frame ~31% of scene
Av Triangles Per Frame		~440K	~207K	~244K	~724K
Memory B/W Per Frame (Mali G72 GPU specific)	VK	652MB (413R + 239W)	268MB (160R + 107W)	135MB (88R + 46W)	73MB (51R + 22W)
	GL	514MB (331R + 182W)	242MB (154R + 87W)

I still think synthetic benchmark testing has a very solid place here – as long as you understand the characteristics of the benchmark. Kishonti’s GFXBench here has been an industry standard for years now, and the new Aztec test gives us a different kind of workload. The new tests are a lot more shader heavy, making use of more complex effects which stress the arithmetic power of the GPUs. While the data in the above table has been collected on an Arm Mali G72 GPU – it still should give an overall indication of what to expect on other architectures. The new tests are also very bandwidth hungry due to their larger textures.

In general games will correlate with benchmarks depending on the percentage of the various graphical workloads, being fillrate or texture heavy, having complex geometries, or simply the ever more increasing complexity of shader effects which demand more arithmetic power of a GPU.

In Aztec Ruins in Normal mode, which is the less demanding new test, the new Apple A12 phones showcase some extremely high peak performance, showcasing a 51% increase over last year’s iPhones.

In terms of sustained performance, the figures quickly reduce after a few minutes and stabilise further down the road. Here, the iPhone XS outperforms the iPhone X by 61%. The Apple A12 is also able to beat the current leader, the Snapdragon 845 inside the OnePlus 6, by 45% in sustained performance.

In the High mode of Aztec Ruins, we’re seeing an eerily similar performance ranking. The iPhone XS’s peak performance is again great, but what should matter is the sustained score. Here again the iPhone XS’s performance is 61% better over the iPhone X. The performance delta to the OnePlus 6’s Snapdragon 845 is reduced to 31% here, which is a tad less than the Normal run, it’s possible we’re hitting some bottlenecks here in some aspects of the microarchitecture.

GPU Power

Platform and GPU power for Apple devices has been something I wanted to publish for some time, but there complexities in achieving this. I was able to get reasonable figures for the new iPhone XS – however data on older SoCs is still something that might have to wait for a future opportunity.

I haven’t had time to measure Aztec across the swath of devices, so we’re still relying on the standard Manhattan 3.1 and T-Rex figures. First off, to get the full performance figures out of the way:

Again in Manhattan 3.1, the new iPhone XS performs an extraordinary 75% better than the iPhone X. The improvements here are not just because of the microarchitectural improvements of the GPU, and having an extra core, all along with the new process node of the SoC, but also thanks to the new memory compression which will reduce power consumption of the external DRAM, something that can represent up to 20-30% of system power in bandwidth heavy 3D workloads. Saved power on the DRAM means more thermal envelope that can be used by the GPU and SoC, increasing performance.

GFXBench Manhattan 3.1 Offscreen Power Efficiency (System Active Power)
	Mfc. Process	FPS	Avg. Power (W)	Perf/W Efficiency
iPhone XS (A12) Warm	7FF	76.51	3.79	20.18 fps/W
iPhone XS (A12) Cold / Peak	7FF	103.83	5.98	17.36 fps/W
Galaxy S9+ (Snapdragon 845)	10LPP	61.16	5.01	11.99 fps/W
Galaxy S9 (Exynos 9810)	10LPP	46.04	4.08	11.28 fps/W
Galaxy S8 (Snapdragon 835)	10LPE	38.90	3.79	10.26 fps/W
LeEco Le Pro3 (Snapdragon 821)	14LPP	33.04	4.18	7.90 fps/W
Galaxy S7 (Snapdragon 820)	14LPP	30.98	3.98	7.78 fps/W
Huawei Mate 10 (Kirin 970)	10FF	37.66	6.33	5.94 fps/W
Galaxy S8 (Exynos 8895)	10LPE	42.49	7.35	5.78 fps/W
Galaxy S7 (Exynos 8890)	14LPP	29.41	5.95	4.94 fps/W
Meizu PRO 5 (Exynos 7420)	14LPE	14.45	3.47	4.16 fps/W
Nexus 6P (Snapdragon 810 v2.1)	20Soc	21.94	5.44	4.03 fps/W
Huawei Mate 8 (Kirin 950)	16FF+	10.37	2.75	3.77 fps/W
Huawei Mate 9 (Kirin 960)	16FFC	32.49	8.63	3.77 fps/W
Huawei P9 (Kirin 955)	16FF+	10.59	2.98	3.55 fps/W

The power figures here are system active power, meaning the total device power, minus the idle power of a given workload scenario (Which includes screen power among other things).

At peak performance, when the device is cool under 22°C ambient temperatures, the Apple A12’s GPU can get quite power hungry, reaching 6W of power. This wasn’t really the peak average of the GPU as I did mention that I saw 3DMark reach around 7.5W (before crashing).

Even at this high power figure, the efficiency of the A12 beats all other SoCs. While this is somewhat interesting, it’s incredibly important to emphasise Apple’s throttling behaviour. After only 3 minutes, or 3 benchmark runs, the phone will throttle by around 25%, to what I describe in the efficiency table as the “Warm” state. Here power reaches reasonable 3.79W. It’s to be noted that the power efficiency did not drastically go up, only improving by 16% over the peak figures. What this could point out is that the platform has a relatively shallow power curve, and performance is mostly limited by thermals.

Moving on to T-Rex, again the iPhone XS showcased a similar 61% improvement in sustained performance.

GFXBench T-Rex Offscreen Power Efficiency (System Active Power)
	Mfc. Process	FPS	Avg. Power (W)	Perf/W Efficiency
iPhone XS (A12) Warm	7FF	197.80	3.95	50.07 fps/W
iPhone XS (A12) Cold / Peak	7FF	271.86	6.10	44.56 fps/W
Galaxy S9+ (Snapdragon 845)	10LPP	150.40	4.42	34.00 fps/W
Galaxy S9 (Exynos 9810)	10LPP	141.91	4.34	32.67 fps/W
Galaxy S8 (Snapdragon 835)	10LPE	108.20	3.45	31.31 fps/W
LeEco Le Pro3 (Snapdragon 821)	14LPP	94.97	3.91	24.26 fps/W
Galaxy S7 (Snapdragon 820)	14LPP	90.59	4.18	21.67 fps/W
Galaxy S8 (Exynos 8895)	10LPE	121.00	5.86	20.65 fps/W
Galaxy S7 (Exynos 8890)	14LPP	87.00	4.70	18.51 fps/W
Huawei Mate 10 (Kirin 970)	10FF	127.25	7.93	16.04 fps/W
Meizu PRO 5 (Exynos 7420)	14LPE	55.67	3.83	14.54 fps/W
Nexus 6P (Snapdragon 810 v2.1)	20Soc	58.97	4.70	12.54 fps/W
Huawei Mate 8 (Kirin 950)	16FF+	41.69	3.58	11.64 fps/W
Huawei P9 (Kirin 955)	16FF+	40.42	3.68	10.98 fps/W
Huawei Mate 9 (Kirin 960)	16FFC	99.16	9.51	10.42 fps/W

Power consumption for T-Rex is in-line with what we saw in Manhattan, with the peak figures on a cold device reaching a little over 6W. After 3 runs, this again reduces to under 4W, at a 28% reduction in performance. Efficiency again doesn’t improve by much here, pointing out to a shallow power curve again.

It’s to be noted that the power measurements of the “Warm” runs don’t represent sustained performance, and I simply wanted to add an additional data-point to the table alongside the peak figures. Sustained power envelopes for most devices are in the 3-3.5W range.

So why does Apple post such big discrepancies between peak performance and sustained performance, when the latter was a keynote focus point for Apple as recent as the iPhone 6 and the A8? The change is due to how everyday GPU use-cases have changed, and how Apple uses the GPU for non 3D related workloads.

Apple makes heavy use of GPU compute for various uses, such as general hardware acceleration in apps to using the GPU compute for camera image processing. These are use-cases where sustained performance doesn’t really matter because they’re transactional workloads, meaning fixed workloads that need to be processed as fast as possible.

Android GPU compute has been a literal disaster over the last few years, and I primarily blame Google for not supporting OpenCL in AOSP – leaving support to be extremely patchy among vendors. RenderScript has never picked up much as it just doesn’t guarantee performance. The fragmentation of Android devices and SoCs has meant that in third-party apps GPU compute is essentially non-existent (Please correct me if I’m wrong!).

Apple’s vertical integration and tight control of the API stack means that GPU compute is a reality, and peak transactional GPU performance is a metric that is worth consideration.

Now while this does explain the throttling, I still do think Apple can do some kind of optimisation in regards to the thermals. I played some Fortnite on the iPhone XS’, and the way that the phones heated up isn’t something that I was very much fan of. Here the must be some kind of way to let actual games and applications which have a characteristic of sustained performance, actually start off with the GPU limited to this sustained performance state.

Other than the thermal and peak performance considerations, the iPhone XS and XS Max, thanks to the new A12 SoC, showcase industry leading performance and efficiency, and currently are the best mobile platforms for gaming, period.

Display Measurement

Apple first introduced OLED panels in the iPhone X last year – and this year’s iPhone XS and XS Max are a continuation of the same designs. The XS’s panel ticks off all the features that are possible to have in a display – OLED, high resolution, wide gamut with colour management, and HDR display with official support of HDR10 and Dolby Vision. The panel is manufactured by Samsung Display, but is said to be a contracted design as blueprinted by Apple.

Among one of the questions I’m still asking myself, is who designed and is providing the display’s DDIC? OLED displays' DDICs are even more important than LCDs', because they not only control colour, but also have to control the active matrix power delivery, and thus the DACs that actually power on the individual pixels.

The iPhone’s display is still a scanning PWM powered panel, meaning the pixels are not actually continuously on, but pretty much work the same way a CRT beam would work – only instead of a single pixel, we have a partial vertical band across the display. The reason for this is just the sheer complexity of running the active-matrix: each subpixel needs to be controlled to 1024 voltage levels to represent the colours of the 10-bit panel. On top of that, the DACs need to have sufficient bit-depth to also provide a seamless range of brightness levels. Here saving on the DAC bit-depth by controlling brightness by PWM is a good workaround the issue.

The iPhone XS’ displays are really excellent at first sight, offering fantastic viewing angles. Personally however, I still have some reservation about the bezel design; Apple has been bested when it comes to screen-to-body ratio by other Android vendors, and I expect to see even more devices come out with what are true full device face screens.

The display’ pixel density doesn’t quite match other 1440p smartphones in terms of sharpness, but it’s still plenty sharp enough for the vast majority of people.

As always, we thank X-Rite and SpecraCal, as measurements are performed with an X-Rite i1Pro 2 spectrophotometer, with the exception of black levels which are measured with an i1Display Pro colorimeter. Data is collected and examined using SpectraCal's CalMAN software.

SpectraCal CalMAN
 XS  :      
XSM:      

In terms of greyscale accuracy, both the iPhone XS and iPhone XS Max present outstanding accuracy, coming in at an astonishing deltaE2000 of 0.79 for the XS and 1.64 for the XS Max. My Max unit seemed to lack intensity in the green channel, which reduced its accuracy score.

Both phones came in very close to the target 6500K of the D65 illumination point, in practice they’re very much perfect white.

Brightness wise, my XS maxed out at 646cd/m², while my XS Max came in at 668cd/m². There is no auto-brightness boost, however at such high brightness levels, there’s no need. Minimum brightness goes down to a little under 2 nits, allowing for comfortable night-time reading.

iPhone XS - iPhone XS Max
SpectraCal CalMAN

If one were to nit-pick, then it’s about the gamma measurement as the XS seemed to undershoot the 2.2 target, resulting in ever so slightly darker images, while my XS Max overshot it, resulting in brighter images. Still both were very much within imperceptible levels, so it’s not a great concern.

iPhone XS - iPhone XS Max
SpectraCal CalMAN

By default, the XS display and software interpret non-wide gamut tagged content as sRGB. Measuring the saturation accuracy here, we see some amazing results from both phones. The XS posted an amazing dE2000 of 0.79 – this is so low that it’s nigh-impossible to get much better, even when manually calibrating a display. The XS Max fared a bit worse at 0.95, but still below 1 which still deserves it the commendation of being excellently accurate.

iPhone XS - iPhone XS Max
SpectraCal CalMAN

When the application supports it, and the media has a wide gamut profile embedded, the iPhone XS displays are able to showcase the higher colour intensities of this wider colour gamut. Apple pretty much standardised “Display P3” in the mobile world – a display mode with the gamut of DCI P3, yet with an identical gamma target of 2.2 of sRGB, ensuring seamless interoperability of both gamuts within a display.

Again, both the iPhone XS and the XS Max showcase outstanding calibration with respective dE2000 of 1.19 and 1.03.

iPhone XS - iPhone XS Max
SpectraCal CalMAN

iPhone XS - iPhone XS Max
SpectraCal CalMAN

The Gretag Macbeth colour targets contain commonly encountered colours, such as skin tones and other colour samples. This test checks not only if the display is able to display the correct colour hue, but also the luminosity.

Again, the iPhones are able to show outstanding figures. The 0.74 score of the iPhone XS is I think the lowest figure we’ve measured on any kind of display, which is amazing. My XS Max figures scored a bit worse, it’s likely that the green channel weakness is part of what’s causing it to be better.

Overall, the iPhone displays are just outstanding. These are the best calibration results we’ve come to measure not only in a smartphone, but likely any display. I have literally nothing negative to say about them, and in terms of picture quality, they are just the best displays on the market.

Display Power

I was curious to see how the new XS fared against last year’s X – as it’s possible there might have been some under-the-hood improvements in terms of panel or emitter materials.

Unfortunately, it looks like the iPhone XS is nigh identical to the iPhone X when it comes to the power characteristics of the panel. My iPhone X had reached just a bit higher brightness and extended up the power curve a bit, but otherwise any differences can just as well be attributed to random manufacturing fluctuations.

Screen Luminance Power Efficiency 100% APL / White @ 200nits
Device	Screen Luminance Power at 200cd/m²	Luminance Power (mW) / Screen area (cm²) Efficiency
LG G7	257 mW	2.93
LG G6	363 mW	4.43
P20	411 mW	4.86
Galaxy S9	563 mW	6.69
P20 Pro	601 mW	6.74
Galaxy S8	590 mW	7.01
iPhone X	~671 mW	~8.31
iPhone XS	~736 mW	~9.11

Comparing the power efficiency at 200cd/m² and normalising the luminance power of the devices for their screen area, we see that the iPhone X and XS fall a tad behind other Samsung OLED panels. I think what this could be attributed to is the 10-bit colour depth of the Apple phones, as their DDIC and the active matrix would need to do more work versus the 8-bit counterparts.

One thing to also very much to take into account is the base power consumption of the phones. The iPhone X, XS and XS Max all fluctuate around 480-500 mW when on a black screen, which is around 150mW more than the iPhone 8 LCD models. This might not sound much, but’s it’s an absolutely huge figure when taking into account that it’s an unavoidable power consumption of the phone whenever the screen is on. I do hope Samsung and Apple alike would be able to focus more on optimising this, as like we’re about to see, it will have an impact on battery life.

Battery Life

The iPhone XS comes with a 2658mAh/10.13Wh battery, while the XS Max has a capacity of 3174mAh/12.08Wh. Again, it’s to be noted that although both phones are quite large form-factor devices by now, Apple’s battery density still largely lags behind the competition. While yes, it’s true that the XS Max’ battery is the biggest that Apple has ever used, it still pales in comparison to the 3500 to 4000mAh that other vendors now employ in the same form-factor.

As we saw in the SPEC analysis, the one advantage that Apple has is an enormous lead in terms of power efficiency of its SoC, which largely makes up for any gap in the battery capacity deficit.

Our web browsing test is a mixed-to-heavy workload that iterates through a set of popular webpages that are hosted on our server. The test loads a web page, pauses, scrolls through it, pauses, and then continues to the next in the set, repeating all over when done. Brightness is fixed at 200cd/m².

The iPhone XS saw a very slight degradation compared to the iPhone X in our test. The 19 minute deficit isn’t terrible, but it does come at a surprise given that Apple had promised improved battery life for the new model. What’s happening is that likely our test is a tad heavier in its workload than what Apple and many other vendors internally test to advertise as the daily battery life of their devices.

The iPhone XS Max came in at 10.3h. Again while this is still good, it’s a degradation over the 11.83h of the iPhone 8 Plus. Here it’s easier to rationalise the difference; the OLED screen of the XS Max is just more power hungry and also has a larger area than the iPhone 8 Plus. Here the increased battery capacity isn’t enough to counteract the panel’s increased needs.

As to why the iPhone XS saw a degradation over the X, I’m not too sure. I did rerun the test on the iPhone X to make sure iOS12 hadn’t impacted the devices – and I got a runtime just 10 minutes lower than what I had tested on the iPhone X back around in January, so the iOS upgrade certainly doesn’t seem to have affected the battery life.

It should be relatively safe to assume that the new A12 should be more efficient in its workloads, even with the increased performance that it brings. One thing that we can’t really verify is the power efficiency at intermediate performance states, as that’s also where CPUs perform a lot of their work at.

We also have to keep in mind the connectivity factor: the new iPhone’s seems to sport a new Broadcom BCM4377 WiFi combo chip which we don’t know much about. Most importantly the new XS have also switched over from a Qualcomm baseband (in our test unit of the iPhone X) to a new Intel XMM7560 baseband.

I’ve generally given up on LTE testing after a few years ago I had run into some serious issues regarding a misconfiguration of my mobile carriers’ baseband stations as they did not have CDRX enabled. This caused an almost 20-30% battery life degradation on Huawei’s devices – and if I hadn’t debugged the issue with HiSilicon I’d probably be none the wiser. Fact is, cellular battery life testing is a lot harder than one would think, and without having a controlled environment, I’m very hesitant to resume cellular battery life testing.

That being said, I will revisit the iPhone X vs iPhone XS battery life topic while on LTE over the weekend and post an update to the review.

Overall, the battery life of the iPhone XS and XS Max are good – they don’t quite reach Apple’s claimed improvements, but that also just might be something that will vary from use-case to use-case.

Camera - Daylight Evaluation: Zoom and Scenic

The iPhone XS’ come with a brand-new camera module: Apple has increased the sensor size of the main camera sensor by upping the pixel pitch from 1.22µm to 1.4µm. This increases the actual area of each pixel by 48%, which means it equivalently increases the light collection capabilities of the sensor.

Apple hadn’t changed the lens system, and because the sensor is now bigger, it means that the effective field of view has increased, reducing the focal length from a 35mm equivalent of 28mm down to 26mm. Apple didn’t really talk much about this during the keynote nor in the marketing materials of the phone, however I do find it as a great positive of the phone, as in everyday shots you just get more scene to work with, something that makes a lot of sense given the continued inclusion of a 2x telephoto lens.

SmartHDR as Apple calls it, is a new HDR system on the new iPhone XS’. The phone will now capture two subsequent pictures, one at high exposure, and one at low exposure, and apply proper processing to create a natural looking high dynamic range image.

[ iPhone XS ] - [ iPhone X ] - [ iPhone 7 ] - [ iPhone 6S ]
[ Galaxy Note9 ] - [ Galaxy S9+ ] - [ Galaxy S8 ]
[ LG G7 ] - [ LG G6 ] - [ LG V30 ]
[ OnePlus 6 ] - [ Mi MIX2S ] - [ Pixel 2XL ]
[ P20 Pro ]

Starting off with the first scene, I’m actually surprised just as how many phones still have trouble with such a shot and in getting a correct photo exposure. The difficulty in this shot in bright sunlight and clear skies, is that a lot of phones will apply their HDR processing and overdo it in terms of bringing down the highlights of the trees, resulting in outright muted colours that just look odd and more something you’d expect from an overcast day.

The iPhone XS performs excellently in this regard, as it’s able to properly maintain the bright sunlit foliage throughout the scene – something the vast majority of other phones had trouble with. Previously I was a fan of the OnePlus 6’s HDR processing, and the V30 also produced great pictures (Something that the G7 couldn’t reproduce).

The improvements of the iPhone XS over the iPhone X are less so in the highlights, but more in the shadows of this shot. The XS just manages to retain a significantly more details in the dark areas.

In terms of detail retention, the iPhone XS might be a bit better than the X. On the left side of the picture I had quite some blurring on the iPhone X, I’m not sure if this is due to shake or optics, something we can verify in later shots.

In terms of the 2x telephoto lens, the iPhone XS sees huge improvements over the iPhone X. While last year’s iPhone had significant issues in terms of colour reproduction on the zoom lens, the iPhone XS’ result is much more akin to the main sensor’s image. There’s an outstanding difference in colours as well as increased detail retention in the shadows. Other 2x zoom phones also have great difficulty in terms of their HDR processing, and it looks like the XS’ new processing puts it at the forefront in this shot.

[ iPhone XS ] - [ iPhone X ] - [ iPhone 7 ] - [ iPhone 6S ]
[ Galaxy Note9 ] - [ Galaxy S9+ ] - [ Galaxy S8 ]
[ LG G7 ] - [ LG G6 ] - [ LG V30 ]
[ OnePlus 6 ] - [ Mi MIX2S ] - [ Pixel 2XL ]
[ P20 Pro ]

Moving onto the next shot, we have again a scenic shot brightly lit up by the sun. Here it’s a lot harder to see the difference between the iPhone X and the new XS – other than the wider field of view of the new module. Both phones do well, although I feel like Apple’s reducing the highlights on the buildings just maybe a bit too much, not properly conveying the brightness of the scene. I feel Samsung is doing better in this regard, but they might overdo it. OnePlus 6’s processing is again a sweet-spot in-between, I feel. 

Again on the left side of the image we see some significant chromatic aberrations on the iPhone X, and the XS is able to provide sharp details up until the very edges of the frame, meaning Apple seems to have improved the quality control on the lens glass.

Detail-wise, the main shooter is on par with the iPhone X and this year’s Galaxy phones, with relatively minor differences between all the phones.

Again on the telephoto lens, the XS improves on the X, even when not quite as evident in this shot. The colour balance is better, and the contrast between highlights and shadows is mode defined.

[ iPhone XS ] - [ iPhone X ] - [ iPhone 7 ] - [ iPhone 6S ]
[ Galaxy Note9 ] - [ Galaxy S9+ ] - [ Galaxy S8 ]
[ LG G7 ] - [ LG G6 ] - [ LG V30 ] - [ OnePlus 6 ]
[ Mi MIX2S ] - [ Pixel 2XL ] - [ P20 Pro ]

We continue on, and again here we see the main improvements of the iPhone XS’ processing is much better retention of shadow detail and contrast.

This is also the first shot where the XS has an evident advantage in detail over the X: The new phone had a mere 1/2257^th second exposure, shorter than the iPhone X’s 1/1621th second one. Beyond the improvements in the shadows, there’s also a lot more detail overall in the scene. Notice how the brickwork on the cathedral tower is a lot more defined on the XS.

In terms of competition, Samsung is able to get even more details out of the shadows and produce a brighter image overall. While the iPhone XS isn’t the best in terms of bringing out the shadows across the phones, it gives an excellent balance between brightness and image sharpness.

[ iPhone XS ] - [ iPhone X ] - [ iPhone 7 ] - [ iPhone 6S ]
[ Galaxy Note9 ] - [ Galaxy S9+ ] - [ Galaxy S8 ]
[ LG G7 ] - [ LG G6 ] - [ LG V30 ] - [ OnePlus 6 ]
[ Mi MIX2S ] - [ Pixel 2XL ] - [ P20 Pro ]

The exposure and colour balance between the XS and X is very similar in this shot. Only again in the shadows does the XS bring out more details. The red chair is a good example of how the wide gamut capture of the iPhone’s can really distinguish themselves from the competition – unfortunately if you don’t have a proper display to showcase this one, it’ll result in toned down colours. Least to say, it looks really iridescent on the iPhone XS’ display.

[ iPhone XS ] - [ iPhone X ] - [ iPhone 7 ] - [ iPhone 6S ]
[ Galaxy Note9 ] - [ Galaxy S9+ ] - [ Galaxy S8 ]
[ LG G7 ] - [ LG G6 ] - [ LG V30 ] - [ OnePlus 6 ]
[ Mi MIX2S ] - [ Pixel 2XL ] - [ P20 Pro ]

In this shot, we again see more shadows than in the iPhone X. The iPhone produces a tad lower exposure than the S9, Note9 and OP6 – however I do understand why this is as the sun against the building really blows out on those phones, but I do think the foliage on the competition looks a bit better. Maybe something in-between would have been the best result. I like the Note9’s processing here a tad more, while detail wise both the phones are neck-in-neck.

[ iPhone XS ] - [ iPhone X ] - [ iPhone 7 ] - [ iPhone 6S ]
[ Galaxy Note9 ] - [ Galaxy S9+ ] - [ Galaxy S8 ]
[ LG G7 ] - [ LG G6 ] - [ LG V30 ] - [ OnePlus 6 ]
[ Mi MIX2S ] - [ Pixel 2XL ] - [ P20 Pro ]

In this shot, while the iPhone XS has more contrast in the shadows, it actually seems to lose a bit of detail versus the iPhone X – something you can see in the darker brickwork on the left bridge. In other parts of the scene, the XS is superior, and resolves a lot more detail in the barks of the trees on the right.

Samsung and OnePlus both had less issues in the shadow-cast bricks, and both provided an overall brighter image, although if that’s a positive is a thing of preference. Personally I do prefer the OP6’s HDR a lot here as it just does a better job on the brickwork.

[ iPhone XS ] - [ iPhone X ] - [ iPhone 7 ] - [ iPhone 6S ]
[ Galaxy Note9 ] - [ Galaxy S9+ ] - [ Galaxy S8 ]
[ LG G7 ] - [ LG G6 ] - [ LG V30 ] - [ OnePlus 6 ]
[ Mi MIX2S ] - [ Pixel 2XL ] - [ P20 Pro ]

Hopefully we’ve had enough sunlit scenes for this page, so moving on to more levelled scenarios.

The very first thing that pops out in this image between the iPhone XS and the iPhone X is that the XS no longer blows out the sky, and we actually see some blue.

While overall exposure and colour balance in the scene is similar, detail retention on the iPhone XS is significantly better. Here the XS is able to resolve cracks and textures on the walls that were invisible to the X.

While the XS stands out against the X in terms of details, it still falls behind the Samsung phones as both the Note9 and S9 do better on the textures of the buildings.

Camera - Daylight - More HDR & Portrait

I wanted to have a second page of daylight photos because I wanted to spend a bit more time and have a tad more varied scenes for to test Apple’s SmartHDR – please enjoy.

[ iPhone XS ] - [ iPhone X ] - [ iPhone 7 ] - [ iPhone 6S ]
[ Galaxy Note9 ] - [ Galaxy S9+ ] - [ Galaxy S8 ]
[ LG G7 ] - [ LG G6 ] - [ LG V30 ] - [ OnePlus 6 ]
[ Mi MIX2S ] - [ Pixel 2XL ] - [ P20 Pro ]

This tunnel was a fun little test – the other end was sunlit while obviously quite dark from my side. I thought this would be a good little visual representation of the raw dynamic ranges that the phones would be able to capture.

Indeed, the iPhone XS is able to go a lot further into the end of the tunnel than the iPhone X, or for that matter, most other phones. This is an extreme show-case of Apple’s new HDR processing and how it’s able to play with bright highlights in scenes.

[ iPhone XS ] - [ iPhone X ] - [ iPhone 7 ] - [ iPhone 6S ]
[ Galaxy Note9 ] - [ Galaxy S9+ ] - [ Galaxy S8 ]
[ LG G7 ] - [ LG G6 ] - [ LG V30 ] - [ OnePlus 6 ]
[ Mi MIX2S ] - [ Pixel 2XL ] - [ P20 Pro ]

One scenario that Apple showcased during the keynote was a shot directly facing the sun. I’ve had users in previous reviews bombard me with comments as to that’s not how you should take a photo. To them I say: that’s an outdated notion of photography.

As computational photography becomes an ever increasingly common theme in devices, we’ll see more and more scenes like this one where shooting against the sun should be no issue at all.

The iPhone XS dramatically improves the shadow detail, and is able to notably reduce the sun’s halo in this shot, but I do think Apple might have overpromised a bit on the notion of computational photography. The best counter-example of this is to just switch over to what the Huawei P20 Pro was able to achieve in its 10MP AI mode, by far surpassing all other phones in the captured dynamic range of the scene. This facet of smartphone photography really opens up a new area of competition, and hopefully we’ll be seeing some exciting things in the future.

[ iPhone XS ] - [ iPhone X ] - [ iPhone 7 ] - [ iPhone 6S ]
[ Galaxy Note9 ] - [ Galaxy S9+ ] - [ Galaxy S8 ]
[ LG G7 ] - [ LG G6 ] - [ LG V30 ] - [ OnePlus 6 ]
[ Mi MIX2S ] - [ Pixel 2XL ] - [ P20 Pro ]

This shot follows the same themes we saw on the previous page, the iPhone XS handles the shadows a lot better and gives a lot more details over the iPhone X.

Samsung again opts for a much brighter picture, but I do think it comes at some cost of detail. Again I think the OnePlus 6’s HDR processing is an excellent middle-ground that would please most people, although Apple has a tad more natural look going for them.

[ iPhone XS ] - [ iPhone X ] - [ iPhone 7 ] - [ iPhone 6S ]
[ Galaxy Note9 ] - [ Galaxy S9+ ] - [ Galaxy S8 ]
[ LG G7 ] - [ LG G6 ] - [ LG V30 ] - [ OnePlus 6 ]
[ Mi MIX2S ] - [ Pixel 2XL ] - [ P20 Pro ]

In less direct sunlit environments, the difference between the iPhone X and XS might not be directly visible the thumbnails, however opening up the full resolution image showcases the XS’s significant increase of detail and textures throughout the whole scene. The larger pixels of the XS sensor along with the deeper DTI (deep trench isolation) results in significantly increased spatial resolution – even though the sensor has the same amount of pixels and even has a wider field of view, resulting in less pixels per given object.

Again Samsung tends for a brighter exposure that I think is a bit too much – detail slightly trails the XS. OnePlus bridges the two vendors in terms of exposure and detail.

[ iPhone XS ] - [ iPhone X ] - [ iPhone 7 ] - [ iPhone 6S ]
[ Galaxy Note9 ] - [ Galaxy S9+ ] - [ Galaxy S8 ]
[ LG G7 ] - [ LG G6 ] - [ LG V30 ] - [ OnePlus 6 ]
[ Mi MIX2S ] - [ Pixel 2XL ] - [ P20 Pro ]

This scene was mostly in the tree shadows, sun sunlit spots coming through the gaps. By now we should understand where the XS’ strengths are: brighter and more defined shadow details.

I think Apple nailed this shot and it has the best balance of exposure as well as the best detail retention. The OP6 closely followed in terms of exposure, but lost in terms of details. Samsung here just overdid it with exposure and just flattens the scene too much.

[ iPhone XS ] - [ iPhone X ] - [ iPhone 7 ] - [ iPhone 6S ]
[ Galaxy Note9 ] - [ Galaxy S9+ ] - [ Galaxy S8 ]
[ LG G7 ] - [ LG G6 ] - [ LG V30 ] - [ OnePlus 6 ]
[ Mi MIX2S ] - [ Pixel 2XL ] - [ P20 Pro ]

Finally the last shot, is again a good showcase of HDR of the different phones. The iPhone XS continues to perform very well here, showing the improvements we’ve seen in previous scenes. Again Samsung is brighter, but slightly loses out on details.

Portrait Mode

[ iPhone XS ] - [ iPhone X ]
[ Galaxy Note9 ] - [ LG G7 ]
[ OnePlus 6 ] - [ Mi MIX2S ] - [ Pixel 2XL ]
[ P20 Pro   ]

Portrait mode is something that’s been quite the rage nowadays, and the iPhone XS promises to take advantage of its new inferencing engine power to create much better separation maps between the foreground subject and the background, to which the computational bokeh effect is applied.

Shooting in portrait mode on most phones means that the actual shot will be taken with the telephoto module, while the wide main camera is also doing work by serving as the depth sensor. Single-module phones such as the Pixel 2 rely solely on the computational power to discern between the subject and the background.

The results on the iPhone XS showcase a significant improvement in the image quality of portrait mode. First of all, the exposure and colour balance of the shot is just significantly better, something that’s universally valid for telephoto shots on the new XS.

The actual bokeh effect on the XS looks to be applied a lot more graduated, and while it’s still possible to see the edge of the pattern in some cases, it’s significantly improved.

Camera - Low Light Evaluation

In low-light scenarios, we should see the new iPhone XS showcase significant improvements thanks to the 50% better light capture ability of the new sensor. Apple’s still only employing a f/1.8 aperture lens on the XS - so while it will improve over past phones, at least on paper it’s still at a disadvantage to say Samsung’s latest phones, which have an extra-wide f/1.5 aperture available to them.

[ iPhone XS ] - [ iPhone X ] - [ iPhone 7 ] - [ iPhone 6S ]
[ Galaxy Note9 ] - [ Galaxy S9+ ] - [ Galaxy S8 ]
[ LG G7 ] - [ LG G6 ] - [ LG V30 ] - [ OnePlus 6 ]
[ Mi MIX2S ] - [ Pixel 2XL ] - [ P20 Pro ]

In this first shot, we immediately see the new iPhone’s advantage over last year’s flagship. There is a lot more definition in the grass, less noise throughout the image, and less blown out lights in the scene.

Unfortunately, Apple is as expected still at a great disadvantage to Samsung here, as the latter is just able to give more light onto the whole scene, and the most evident, more colour to the grass. In terms of raw low light capture, the Huawei P20 Pro is still far ahead here, thanks to its massive sensor that is able to collect significantly more light.

[ iPhone XS ] - [ iPhone X ] - [ iPhone 7 ] - [ iPhone 6S ]
[ Galaxy Note9 ] - [ Galaxy S9+ ] - [ Galaxy S8 ]
[ LG G7 ] - [ LG G6 ] - [ LG V30 ] - [ OnePlus 6 ]
[ Mi MIX2S ] - [ Pixel 2XL ] - [ P20 Pro ]

At first glance, the iPhone XS didn’t shoot a much brighter picture than the iPhone X in this construction scene. Opening up the full resolution images however shows that the new XS showcases much better details and lower noise. It’s not enough to compete with the S9+, and certainly not with the insane ISO25600 shot of the P20 Pro.

It’s interesting to see the improvements over the years from the iPhone 6S on – which barely manages to capture anything in this scene.

[ iPhone XS ] - [ iPhone X ] - [ iPhone 7 ] - [ iPhone 6S ]
[ Galaxy Note9 ] - [ Galaxy S9+ ] - [ Galaxy S8 ]
[ LG G7 ] - [ LG G6 ] - [ LG V30 ] - [ OnePlus 6 ]
[ Mi MIX2S ] - [ Pixel 2XL ] - [ P20 Pro ]

The next shot is probably the only one that I found to be really problematic for Apple. Both on the iPhone X and the new XS, the resulting images weren’t consistent in consecutive shots. In four shots in a row, the iPhone XS kept changing the colour temperature. The same thing happened on the iPhone X, so I think this was part of Apple’s exposure / colour balance algorithm.

Colour balance aside, the exposure is similar between the X and the XS, and all the improvements of the new sensor go directly into improved detail and noise reduction throughout the scene, which is significantly better again compared to last year’s iPhone.

Here Apple is very close to Samsung, showcasing a bit better shadows, but still losing out in details in some parts of the scene. The P20 Pro is yet again the low-light kind here, as it just have that much more dynamic range work with.

[ iPhone XS ] - [ iPhone X ] - [ iPhone 7 ] - [ iPhone 6S ]
[ Galaxy Note9 ] - [ Galaxy S9+ ] - [ Galaxy S8 ]
[ LG G7 ] - [ LG G6 ] - [ LG V30 ] - [ OnePlus 6 ]
[ Mi MIX2S ] - [ Pixel 2XL ] - [ P20 Pro ]

Again, the iPhone’s new sensor comes into play in these concrete trucks. The XS makes very good dealing of the blown highlights present in the iPhone X shot. Samsung is able to produce more vibrancy in the blue of the trucks. Huawei’s multi-exposure computational photography night mode is the best of all phones here as it’s just able to bring out that much more from the shadows.

[ iPhone XS ] - [ iPhone X ] - [ iPhone 7 ] - [ iPhone 6S ]
[ Galaxy Note9 ] - [ Galaxy S9+ ] - [ Galaxy S8 ]
[ LG G7 ] - [ LG G6 ] - [ LG V30 ] - [ OnePlus 6 ]
[ Mi MIX2S ] - [ Pixel 2XL ] - [ P20 Pro ]

Apple's use of SmartHDR in this picture is extremely evident, as it really brings down the highlights of the lamp and brings out more shadows throughout the scene. The XS provides better detail, but it’s not as big of a difference as we’ve seen in other shots.

Apple’s usage of HDR here puts it ahead of the Samsung devices, trading blows with the P20 Pro, winning in some regards, while losing in others.

[ iPhone XS ] - [ iPhone X ] - [ iPhone 7 ] - [ iPhone 6S ]
[ Galaxy Note9 ] - [ Galaxy S9+ ] - [ Galaxy S8 ]
[ LG G7 ] - [ LG G6 ] - [ LG V30 ] - [ OnePlus 6 ]
[ Mi MIX2S ] - [ Pixel 2XL ] - [ P20 Pro ]

Finally, I wanted to test the iPhone XS to its limits and see what it can do in essentially impossible scenarios of low light.

Camera Video Recording

In terms of video recording, the iPhone XS promises an improved dynamic range in modes up to 30fps. What this likely means is that the phone’s able to capture in HDR mode in the 30fps modes, doing the same kind of processing we also see in SmartHDR still pictures.

Also something I’ve dreaded on iPhones for years; the new iPhone XS finally introduces stereo audio recording. Why it took Apple such a long time to finally introduce stereo recording is something that boggles the mind, but, let’s not complain, as we now finally have it on the new generation.

 

iPhone XS:      iPhone X: 

Comparing the iPhone XS video to the iPhone X, there’s one thing that is immediately very evident: the new XS is able to produce much better image stabilisation than last year’s flagship. Indeed, it looks like Apple vastly improved the OIS/EIS on the new phones, as the wobble that happens when walking is gone on the XS.

Audio recording finally is up to par, and we can hear the wind and rustling leaves of the trees around us. I think Apple might still have to work a bit on the wind noise cancellation, as in some parts the audio sounded as if it was inside a tube.

In terms of image quality, Apple’s claims of the improved dynamic range are very much verified. The phone showcases a lot more brought down highlights in the scene, and in darker areas, show better shadows. It’s unfortunate that this is limited only to the 30fps modes, but it’s understandable.

Switching over from the main lens to the telephoto lens happens relatively fast, although with a short exposure flash and a slight delay on the first zoom. 4K60 recording doesn’t allow for the use of the telephoto lens.

All in all, the video recording quality of the new iPhone XS is massively improved in all areas of stabilisation, picture quality, and audio. 4K30 recording on the XS is probably the best I’ve seen on any smartphone – a definitive applause to Apple for the improvements here.

Speaker Evaluation

Apple claimed to have improved the speaker audio quality on the new iPhone XS, allowing for more stereo separation and filling sound. I had introduced a new speaker evaluation method a few months ago because this year’s efforts by smartphone vendors to improve speaker quality has been very pronounced, and I wanted to have a way to objectively convey these improvements.

Starting off with speaker loudness, we’re measuring the phones at maximum volume, both in one-hand portrait mode, as well as two-handed mode where the palms are cupped towards the user. These two use-cases are what I find myself most often using the phone’s speakers in, so hopefully that also represents how most users use it as well, please let me know otherwise!

Using a pink noise signal, the iPhone XS pretty much falls into line with the results of the iPhone X, coming in at a very loud 82.8dBA in portrait mode and 87.6dbA in two-handed mode. Apple’s sound directionality on the iPhone X and XS is among the best, most likely due to the fact that the stereo earpiece is among the loudest of current generation smartphones.

Measuring the frequency response of the speakers, we see the iPhone XS closely following the measurement of the iPhone X, however there’s a major difference in the mid-range where the XS is around 5dB louder, raising instrumental frequencies and voices. This difference is what I think Apple is referring to when talking about better “fullness”, as it is evident when playing back media.

To better demonstrate the difference between the phones, I’ve attempted to capture them with a binaural microphone setup. Now I know my environment isn’t perfect as I don’t have the necessary sound dampening equipment, but I hope it does serve as an overall adequate A/B comparison between the phones. I’ve tried to calibrate the sound as much as possible recorded by the setup to a flat frequency response, although I’m sure there are improvements to be made. As a comparison, I also included calibrated speakers as a baseline to get an idea of the microphone setup.

The audio is meant to be listened to with headphones, or even better with IEMs, as this will give the intended playback of the binaural recording.

The iPhone XS’ improvements in the mid-range are quite evident as voices sound deeper and more pronounced on the new phone. Stereo separation is also quite good – resulting in a filling audio experience.

I included the S9+ and G7 as comparison devices. Samsung still does a significantly better job at the low-mid ranges which gives the phone more overall presence than the iPhones, also has an advantage in the very high frequencies giving more clarity, however the new iPhone’s XS strength point in the mid-ranges is the S9’s weakness, and vocals sound a lot less present than on the XS.

As for the G7, I just wanted to showcase a mono speaker device, and just how huge the audio difference is. Unfortunately the G7, even though it promises to have a good speaker, fails in practice.

Conclusion & End Remarks

While the iPhone XS and XS Max in one sense are just another iteration on last year’s iPhone X, they’re also a big shift for Apple’s line-up. Rather than being actual successors to the iPhone 8 and 8 Plus they're closer to next-generation replacements, but with some significant differences. In that respect I do regret missing out on the iPhone XR for this review, as I think it’s going to be an incredibly attractive alternative to the XS models.

Design wise, there’s not much to talk about the XS: the smaller variant is nigh identical to the iPhone X, with the only visual differences between the phones being the added antenna lines on the XS, virtue of the new 4x4 MIMO cellular capabilities of the phones.

The XS Max sports Apple’s biggest screen, and in a sense I do like the design more because it does have a bigger screen-to-body ratio. Apple’s bezel design is intentional, but I did hope they had shaved 1-2mm off the sides, as I’ve gotten used to other, more full-screen devices. One thing to consider about the XS Max, is that’s it’s really heavy for a phone, passing the 200g mark at 208g.

The screens of the XS and XS Max are the best displays among any devices on the market: While Samsung still has a density advantage, the Apple phones just outgun competing phones in terms of colour accuracy and picture quality. The 10-bit panel allows seamless colour management between sRGB and Display P3 modes depending on content, and Apple’s still the only vendor able to do this without having significant drawbacks.

The Apple A12 is a beast of a SoC. While the A11 already bested the competition in terms of performance and power efficiency, the A12 doubles down on it in this regard, thanks to Apple’s world-class design teams which were able to squeeze out even more out of their CPU microarchitectures. The Vortex CPU’s memory subsystem saw an enormous boost, which grants the A12 a significant performance boost in a lot of workloads. Apple’s marketing department was really underselling the improvements here by just quoting 15% - a lot of workloads will be seeing performance improvements I estimate to be around 40%, with even greater improvements in some corner-cases. Apple’s CPU have gotten so performant now, that we’re just margins off the best desktop CPUs; it will be interesting to see how the coming years evolve, and what this means for Apple’s non-mobile products.

On the GPU side, Apple’s measured performance gains are also within the promised figures, and even above that when it comes to sustained performance. The new GPU looks like an iteration on last year’s design, but an added fourth core as well as the important introduction of GPU memory compression are able to increase the performance to new levels. The negative thing here is I do think Apple’s throttling mechanism needs to be revised – and by that I mean not that it shouldn’t throttle less, but that it might be better if it throttled more or even outright capped the upper end of the performance curve, as it’s extremely power hungry and does heat up the phone a lot in the initial minutes of a gaming session.

On the camera side, Apple made some very solid improvement all-around. The new sensor’s increased pixel size allows for 50% more light sensitivity, but the improved DTI of the sensor also allows for significantly finer details in bright conditions, essentially increasing the effective spatial resolution of the camera. SmartHDR works as promised, and it’s able to produce images with improved dynamic range. The telephoto lens is the one use-case where the XS really stands out over the iPhone X as exposure and colour rendition are significantly improved, one of the weak points of many telephoto cameras nowadays. Overall in daylight, the new iPhone is easily among the best smartphone cameras on the market.

In low light the iPhone XS also sees a big improvement, however it’s not enough to quite match Samsung’s hardware and Huawei’s processing. I do hope Apple will make use of the newfangled computational photography in more use-cases, as we’re seeing some great innovation from the competition in this regard.

Video recording of the iPhone XS is also a major improvement of the phone. From better dynamic range, better stabilisation, to better and now stereo audio recording, Apple makes a significant leap in the video performance of the new iPhones.

In terms of battery life, it was surprising that the iPhone XS wasn’t much of an upgrade over the iPhone X in our test. I’m still not sure if this is something related to some sort of hidden inefficiency of the A12, or maybe something to do with the new WiFi or cellular modem. For the latter, we’ll be revisiting the topic shortly, and to also re-validate the battery life numbers of this review.

For the iPhone XS Max, I wasn’t surprised to see battery life be less than on the iPhone 8 Plus – the OLED screen is less efficient than the LCD display of last year’s phone – and the increased battery capacity is not enough to counter-act this. It’s just something to keep in mind for the big-phone users out there eyeing the iPhone XS Max in particular.

Overall, are the new iPhones worth it to upgrade to? If you’re an iPhone X user, I think my answer is no. If you’re coming from an older device, then my answer is… wait it out. When having a hands-on with the XR at the keynote event, my first thought was that this would be the model that would see the most success for Apple this generation. The problem here is that Apple is asking for a lot of money – if you’re entrenched in the iOS ecosystem, I think it’s best to evaluate the individual pros and upgrades that the new iPhone XS brings over your current device.

The value proposition aside, the new iPhone XS and XS Max are, as always, extremely polished devices, and the best phones that Apple has released to date.