Intro - All right, so we wanted to answer the question once and for all. What is the best smartphone camera out right now? Maybe you think it's the iPhone. Or maybe it's a Samsung flagship. Or maybe it's one of those Pixels. Now, in years past, we've done a blind voting using social media polls. Which was fun and yielded some surprising results. But that would still depend on the match-ups that I planted at the very beginning. So I specifically put the iPhone and the Pixel on opposite sides of the bracket expecting the best ones to hopefully meet in the finals. But they never did. And there's always a bit of human input. It's possible I could have accidentally Put the second-best phone up against the best phone in the first round and it got eliminated. So this time we're gonna solve all that once and for all. Welcome to the "Blind Smartphone Camera Test: Scientific Edition. " So here's the idea. We took 16 smartphones that came out during this year. The flagships, your expected heavy hitters, but then also some mid-rangers as well, all the way down to the unexpected. And then we took the same photo with each one of them. Harder than it sounds because you have to hold perfectly still for several minutes straight. But we did this for three different types of photos. a standard daylight photo, a low light photo, and a portrait mode snap.
And then we take all these images, compress them, strip them of their EXIF data, and assign each one a letter. Now for maximum statistical confidence, we would have to ask you guys to vote on every single possible match-up at least once. So A versus B, then A versus C, then A versus D, all the way through. And then B versus C, B versus D. Over and over. Sixteen chose two for my stat heads. But that's a bit too much. I mean, we wanna be scientific, but having you sit there and vote over 100 times for each category was a bit of a tall ask. We Built a Voting Site So instead, we got a little clever with it. And we built a power ranking system, actually an ELO rating algorithm, which you might have heard of. They use this in chess, they use this in table tennis. And, in any video game, you've played where there's a bunch of different match-ups all happening at once and you need to form a ranking of them. But this time we'll be using it to sort of form a power ranking of smartphone cameras. So basically, when you hit enough sanctioned match-ups, you're given a rating based on the ratings of your match-ups. So you, along with everybody else, Get shown comparisons at random and we have this little progress bar along the bottom. You vote for the better photo each time, and then after a little while, We have enough information to determine your individual, customized, statistically significant winner. So we built this whole site
with the ELO rating system in there. We loaded up all the photos And then we tossed this first into our Discord to have some of you beta-test this before anyone else knew it existed. So we could experiment with the data formatting on the out end so we could make sure it was all organized. So actually shout out to Discord for sponsoring this portion of today's video. So with the launch of their new Server Subscription feature, We opened up an exclusive, premium area on our Discord server where members can ask the team, including me, you know, general questions about tech or any of the content and get exclusive insider info that you won't find anywhere else like this test. And also, if you're wondering, Discord does it right with a revenue split. It's 90-10, so 90% going to the creator. So your subscription directly supports us. And if you're a creator looking for more information on this particular feature and adding it to your own Discord, Check out the link below. So anyway, I dropped the site in there, A bunch of you got in there quickly and voted, and basically, we got the data back in this text format. It looks kind of messy at first, but actually, it's super helpful and we can work with this. A summary of how many times every single match-up got voted on and the votes for each side. Also since I had a lot of help with building this site and loading up all the pictures, Even though I know which phones were involved, I don't know which phones took which pictures. So I could blind test myself. And it was every bit as engaging as I had hoped. I could kind of tell some of the phones
that I think took certain pictures. But for the most part, I would just take a glance, and if I see those two pictures side by side on Instagram or Twitter, I'm just picking the better one. No pixel peeping or thorough inspection. It's just a picture on the internet, vote. And in the end, my standard picture winner was G, My light winner was K, and my portrait mode winner was also K. So everything looked like it was working. The data processing was working as we wanted. So the only thing left to do was send it live, and here we go. So as soon as it went public, you guys jumped right on it. It peaked at around 35,000 votes per minute and then ended up totaling more than 21 million votes in about three days. There were essentially zero bad actors. And now we have a load of data. Revealing the Phones Many of you may have gone through this, which means you've done the voting And now you have your letter. This means it's time to reveal all of the smartphones that were involved in this test. And here you go, A through P. That's all 16 phones right there in their order. Which means the winner when I blind took the test myself for my standard photo was the Pixel 6A. My winner for portrait mode and night mode was the Pixel 7. But what were the overall winners according to statistics, over the 20-plus million votes according to science? The Winners and Losers I don't know if there's any other collection of data like this anywhere on the internet. So this is super cool to get into.
So let's dig into it. So first of all, the overall winner so with the highest average ELO rating across all three categories. In third place is the Asus Zenfone 9. In second place is the Pixel 7 Pro. And first place is the Pixel 6A, the people's choice camera of 2022. But there is a lot of fascinating information to unpack here. So first of all, doing well in one category did not necessarily guarantee doing well across any of the other categories, right? So the Pixel 7 Pro came in second, fourth, and first in the three categories. The Pixel 6A came in third, second, and second. But the Oppo Find X5 Pro came in first for standard, then third for low light, and 15th for portrait mode. This means that if you wanted, You could argue for the importance of the standard photo over the low light or the portrait mode photos. So let's say the standard photo is like three times as important to most people as the other two categories, which feels about right. With that weighted importance on standard photos, Then the Oppo Find X5 Pro becomes the overall winner which is interesting. And in case you're wondering where the iPhone was in all this, The iPhone 14 Pro tested in sixth place, 10th place, and fifth place. But in natural curiosity, for me anyway, was to ask what were the biggest losers. Which ones got destroyed the most in this test? And your undisputed winners of losing the most are the Sony Xperia 1 IV and the Moto Edge 30 Ultra.
These phones performed terribly in these tests. So the Sony, despite its world-class sensor and extremely advanced apps, went in full auto, like the rest of these phones, did not produce good pictures and ended up with the lowest average ELO rating across all three categories. And then the Moto Edge 30 Ultra actually had the most overall losses. It took a bizarre sequence of photos, including an extremely dark standard light photo and an incredibly sharpened low-light photo, that it's the only one to have 2 million total votes against it. Now here's another interesting thought. Votes per dollar. If we wanted to sort by bang for the buck basically Which ones, if we account for the launch price, the MSRP, Which one was performing the best relative to its price? And the two winners for votes per dollar by far are the Pixel 6A. No surprise. Over 4,200 votes per dollar. And the Realme 10 Pro+ also sneaks in there, the only other phone up over 4,000 votes per dollar. Sneaky good camera phone for $379. And then the obvious losers, again, are the $900 Moto Edge and the $1,600 Sony Xperia 1 IV. Also, I just had to. I had to look through the text files and find the biggest dunk. Just the biggest overall match-up discrepancy between two phones in any category. And I found it. That would be the portrait mode Pixel 7 Pro versus Sony Xperia 1 IV with the Pixel winning 98% of the votes here. So to the 1,523 of you who voted for Sony here,
not sure what you saw that the rest of us didn't. What Happened But overall, just more than seeing which ones got the most votes is seeing why certain phones and certain pictures got the most votes. And the best way to do that was just putting all of the photos in order side-by-side in order of how many votes they got. And that's how we started to see Some really interesting patterns. So like for example, last time in the head-to-head, We kind of established that in the case of A versus B, If one vote is a little brighter than the other, It'll probably win. But how far can that go? Is there a such thing as too bright in this case? And it turned out, yes. This is all the standard mode photos in order. And yes, the three darkest photos are the three biggest losers. Hands down, they crushed any shadow detail in my hair and the extremely comfortable sweater I'm wearing on shop. mkbhd. com. But right above that is three of the absolute brightest photos. Like these were just overexposed straight up. Matter of fact, it looks like we kind of got this order of great photos, then too bright, then too dark. So it shows me that good exposure wins. But if you're gonna miss, Then yeah, people would rather have it be too bright than too dark. With night mode altogether and in order, you can kind of see a bit of the same thing. So the most neutral, correct exposures generally won if they were sharp.
This time there are about four good exposures. Vivo X80 Pro, Pixel 6A, Oppo Find X5 Pro, and Pixel 7 Pro. Then there's the two brights, which people mostly preferred over the two darks. Also, there were some seriously wild things happening with some of these photos in low light. The Vivo, I mean, it nailed it. It's the low-light king. The S22 ultra interestingly came in fifth place even when completely missing focus on my stationary face and hitting the background for some reason. The iPhone 14 Pro did an enormous amount of HDR That had me looking kind of green like a zombie, But it still came in 11th place here. The Moto Edge Plus, as I mentioned, kicked out maybe the most over-sharpened photo the world has ever seen. And then in dead last is the iPhone SE, the only phone in this test that does not have night mode. And the sad thing is that it does have OIS, It does have a decent sensor, and it does have a new processor. But Apple just feels the need to protect their more expensive phones with more premium features. So they didn't give the SE night mode, and it suffered big time here. Then in portrait mode, we have the most variety of them all because not only are we judging things that we normally would about a photo like sharpness, color exposure, et cetera, but we are also now adding cutout quality and artificially blurred background quality. And every phone uses a slightly different focal length for their portrait mode by default. Some of them do a 2x portrait mode, some of them 2. 5, some of them 3x, some of them even do 1x.
But combining all of these factors, the Pixel 7 Pro, absolutely dominated portrait mode. This was the runaway winner here in this category. Great cutout, great exposure, great colors, and a nice soft background. It nailed it, but guess what? The Pixel 6A was in second and the Realme 10 Pro+ snuck in there in third. Right above what I consider the best cutout of the entire competition, which came from this Galaxy S22 Ultra. I think it's pretty close to perfect at edge detection here. Pretty similar to the Zenfone 9's cutout, but a better overall photo. So it got fourth place instead of seventh. I was also curious if there would be a correlation between how many votes a phone got and whether or not they used more of a telephoto focal length. Just because maybe it would feel more natural to have a blurred background if you're at 3x. But when we lined it all up, there was almost no trend line at all and no correlation. So that was interesting. The only other definite thing I noticed about portrait mode photos is that the three worst photos that got the fewest votes by far were also by far the brightest photos. So see, it turns out you can overdo the brightness. What we Learned And that, I think is it. So summary time, just from all the data that we have. Things that we learned. Number one is that being good in one of these categories did not necessarily guarantee being good at the others. That's reflected in the data. Two, is that Pixel 6A and the Realme 10 Pro+
turned out to be the best bang for the buck As far as a smartphone camera from the phones that came out in 2022. And the Pixel 6A is the overall winner matches with it winning our bracket-style social media polling blind test from last year, which the Pixel 5A won. Last but not least, Yeah, brighter photos do typically win, but you can be too bright. So I think this mission was a success. We learned a lot. I'll say congrats to the Pixel 6A on the People's Choice Award, so to speak. And you know what? We're gonna leave up the voting website till, let's say, through the end of this year. Till the end of January how about? So January 31st is the last day you'll be able to go to vote. mkbhd. com and blind-test yourself. And I think we'll replace the letters now with the actual names of the phones Now that it's been revealed. But we don't wanna have to pay for all these terabytes of data processing forever. So that's your deadline. So thanks to y'all on Discord for helping us beta test this thing. Thanks to Zach for helping us build this site and implement the ELO-style power ranking system.