Intro
- All right, so we wanted to answer the question
once and for all.
What is the best smartphone camera out right now?
Maybe you think it's the iPhone.
Or maybe it's a Samsung flagship.
Or maybe it's one of those Pixels.
Now, in years past, we've done a blind voting
using social media polls.
Which was fun and yielded some surprising results.
But that would still depend on the match-ups
that I planted at the very beginning.
So I specifically put the iPhone
and the Pixel on opposite sides of the bracket
expecting the best ones to hopefully meet in the finals.
But they never did.
And there's always a bit of human input.
It's possible I could have accidentally
Put the second-best phone up against the best phone
in the first round and it got eliminated.
So this time we're gonna solve all that once and for all.
Welcome to the "Blind Smartphone Camera Test:
Scientific Edition."
So here's the idea.
We took 16 smartphones that came out during this year.
The flagships, your expected heavy hitters,
but then also some mid-rangers as well,
all the way down to the unexpected.
And then we took the same photo
with each one of them.
Harder than it sounds
because you have to hold perfectly still
for several minutes straight.
But we did this
for three different types of photos.
a standard daylight photo, a low light photo,
and a portrait mode snap.
And then we take all these images, compress them,
strip them of their EXIF data, and assign each one a letter.
Now for maximum statistical confidence,
we would have to ask you guys to vote
on every single possible match-up at least once.
So A versus B, then A versus C, then A versus D,
all the way through.
And then B versus C, B versus D. Over and over.
Sixteen chose two for my stat heads.
But that's a bit too much.
I mean, we wanna be scientific,
but having you sit there and vote over 100 times
for each category was a bit of a tall ask.
We Built a Voting Site
So instead, we got a little clever with it.
And we built a power ranking system,
actually an ELO rating algorithm,
which you might have heard of.
They use this in chess, they use this in table tennis.
And, in any video game, you've played
where there's a bunch of different match-ups
all happening at once
and you need to form a ranking of them.
But this time we'll be using it to sort of
form a power ranking of smartphone cameras.
So basically, when you hit enough sanctioned match-ups,
you're given a rating based
on the ratings of your match-ups.
So you, along with everybody else,
Get shown comparisons at random
and we have this little progress bar along the bottom.
You vote for the better photo each time,
and then after a little while,
We have enough information
to determine your individual, customized,
statistically significant winner.
So we built this whole site
with the ELO rating system in there.
We loaded up all the photos
And then we tossed this first into our Discord
to have some of you beta-test this
before anyone else knew it existed.
So we could experiment with the data formatting
on the out end so we could make sure it was all organized.
So actually shout out to Discord
for sponsoring this portion of today's video.
So with the launch of their new Server Subscription feature,
We opened up an exclusive, premium area
on our Discord server where members can ask the team,
including me, you know, general questions about tech
or any of the content and get exclusive insider info
that you won't find anywhere else like this test.
And also, if you're wondering,
Discord does it right with a revenue split.
It's 90-10, so 90% going to the creator.
So your subscription directly supports us.
And if you're a creator looking for more information
on this particular feature
and adding it to your own Discord,
Check out the link below.
So anyway, I dropped the site in there,
A bunch of you got in there quickly and voted,
and basically, we got the data back in this text format.
It looks kind of messy at first, but actually,
it's super helpful and we can work with this.
A summary of how many times
every single match-up got voted on
and the votes for each side.
Also since I had a lot of help with building this site
and loading up all the pictures,
Even though I know which phones were involved,
I don't know which phones took which pictures.
So I could blind test myself.
And it was every bit as engaging as I had hoped.
I could kind of tell some of the phones
that I think took certain pictures.
But for the most part, I would just take a glance,
and if I see those two pictures side by side
on Instagram or Twitter,
I'm just picking the better one.
No pixel peeping or thorough inspection.
It's just a picture on the internet, vote.
And in the end, my standard picture winner was G,
My light winner was K,
and my portrait mode winner was also K.
So everything looked like it was working.
The data processing was working as we wanted.
So the only thing left to do was send it live,
and here we go.
So as soon as it went public, you guys jumped right on it.
It peaked at around 35,000 votes per minute
and then ended up totaling more than 21 million votes
in about three days.
There were essentially zero bad actors.
And now we have a load of data.
Revealing the Phones
Many of you may have gone through this,
which means you've done the voting
And now you have your letter.
This means it's time to reveal all of the smartphones
that were involved in this test.
And here you go, A through P.
That's all 16 phones right there in their order.
Which means the winner when I blind took the test myself
for my standard photo was the Pixel 6A.
My winner for portrait mode and night mode
was the Pixel 7.
But what were the overall winners according to statistics,
over the 20-plus million votes according to science?
The Winners and Losers
I don't know if there's any other collection of data
like this anywhere on the internet.
So this is super cool to get into.
So let's dig into it.
So first of all, the overall winner
so with the highest average ELO rating
across all three categories.
In third place is the Asus Zenfone 9.
In second place is the Pixel 7 Pro.
And first place is the Pixel 6A,
the people's choice camera of 2022.
But there is a lot of fascinating information
to unpack here.
So first of all, doing well in one category
did not necessarily guarantee doing well
across any of the other categories, right?
So the Pixel 7 Pro came in second, fourth,
and first in the three categories.
The Pixel 6A came in third, second, and second.
But the Oppo Find X5 Pro came
in first for standard, then third for low light,
and 15th for portrait mode.
This means that if you wanted,
You could argue for the importance of the standard photo
over the low light or the portrait mode photos.
So let's say the standard photo is like three times
as important to most people as the other two categories,
which feels about right.
With that weighted importance on standard photos,
Then the Oppo Find X5 Pro becomes the overall winner
which is interesting.
And in case you're wondering
where the iPhone was in all this,
The iPhone 14 Pro tested in sixth place,
10th place, and fifth place.
But in natural curiosity, for me anyway,
was to ask what were the biggest losers.
Which ones got destroyed the most
in this test?
And your undisputed winners of losing the most
are the Sony Xperia 1 IV and the Moto Edge 30 Ultra.
These phones performed terribly in these tests.
So the Sony, despite its world-class sensor
and extremely advanced apps,
went in full auto, like the rest of these phones,
did not produce good pictures
and ended up with the lowest average ELO rating
across all three categories.
And then the Moto Edge 30 Ultra actually
had the most overall losses.
It took a bizarre sequence of photos,
including an extremely dark standard light photo
and an incredibly sharpened low-light photo,
that it's the only one to have 2 million total votes
against it.
Now here's another interesting thought.
Votes per dollar.
If we wanted to sort by bang for the buck basically
Which ones, if we account for the launch price, the MSRP,
Which one was performing the best relative to its price?
And the two winners for votes per dollar by far
are the Pixel 6A.
No surprise. Over 4,200 votes per dollar.
And the Realme 10 Pro+ also sneaks in there,
the only other phone up over 4,000 votes per dollar.
Sneaky good camera phone for $379.
And then the obvious losers, again, are the $900 Moto Edge
and the $1,600 Sony Xperia 1 IV.
Also, I just had to.
I had to look through the text files
and find the biggest dunk.
Just the biggest overall match-up discrepancy
between two phones in any category.
And I found it.
That would be the portrait mode Pixel 7 Pro
versus Sony Xperia 1 IV
with the Pixel winning 98% of the votes here.
So to the 1,523 of you who voted for Sony here,
not sure what you saw that the rest of us didn't.
What Happened
But overall, just more than seeing which ones
got the most votes is seeing why certain phones
and certain pictures got the most votes.
And the best way to do that
was just putting all of the photos
in order side-by-side in order of how many votes they got.
And that's how we started to see
Some really interesting patterns.
So like for example, last time in the head-to-head,
We kind of established that in the case of A versus B,
If one vote is a little brighter than the other,
It'll probably win.
But how far can that go?
Is there a such thing as too bright in this case?
And it turned out, yes.
This is all the standard mode photos in order.
And yes, the three darkest photos
are the three biggest losers.
Hands down, they crushed any shadow detail in my hair
and the extremely comfortable sweater I'm wearing
on shop.mkbhd.com.
But right above that
is three of the absolute brightest photos.
Like these were just overexposed straight up.
Matter of fact, it looks like we kind of got this order
of great photos, then too bright, then too dark.
So it shows me that good exposure wins.
But if you're gonna miss,
Then yeah, people would rather have it be too bright
than too dark.
With night mode altogether and in order,
you can kind of see a bit of the same thing.
So the most neutral, correct exposures generally won
if they were sharp.
This time there are about four good exposures.
Vivo X80 Pro, Pixel 6A, Oppo Find X5 Pro, and Pixel 7 Pro.
Then there's the two brights,
which people mostly preferred over the two darks.
Also, there were some seriously wild things happening
with some of these photos in low light.
The Vivo, I mean, it nailed it.
It's the low-light king.
The S22 ultra interestingly came in fifth place
even when completely missing focus on my stationary face
and hitting the background for some reason.
The iPhone 14 Pro did an enormous amount of HDR
That had me looking kind of green like a zombie,
But it still came in 11th place here.
The Moto Edge Plus, as I mentioned,
kicked out maybe the most over-sharpened photo
the world has ever seen.
And then in dead last is the iPhone SE,
the only phone in this test that does not have night mode.
And the sad thing is that it does have OIS,
It does have a decent sensor,
and it does have a new processor.
But Apple just feels the need
to protect their more expensive phones
with more premium features.
So they didn't give the SE night mode,
and it suffered big time here.
Then in portrait mode, we have the most variety of them all
because not only are we judging things
that we normally would about a photo
like sharpness, color exposure, et cetera,
but we are also now adding cutout quality
and artificially blurred background quality.
And every phone uses a slightly different focal length
for their portrait mode by default.
Some of them do a 2x portrait mode, some of them 2.5,
some of them 3x, some of them even do 1x.
But combining all of these factors,
the Pixel 7 Pro, absolutely dominated portrait mode.
This was the runaway winner here in this category.
Great cutout, great exposure,
great colors, and a nice soft background.
It nailed it, but guess what?
The Pixel 6A was in second
and the Realme 10 Pro+ snuck in there in third.
Right above what I consider the best cutout
of the entire competition,
which came from this Galaxy S22 Ultra.
I think it's pretty close to perfect at edge detection here.
Pretty similar to the Zenfone 9's cutout,
but a better overall photo.
So it got fourth place instead of seventh.
I was also curious if there would be a correlation
between how many votes a phone got
and whether or not they used more
of a telephoto focal length.
Just because maybe it would feel more natural
to have a blurred background if you're at 3x.
But when we lined it all up,
there was almost no trend line at all and no correlation.
So that was interesting.
The only other definite thing I noticed
about portrait mode photos
is that the three worst photos that got the fewest votes
by far were also by far the brightest photos.
So see, it turns out you can overdo the brightness.
What we Learned
And that, I think is it.
So summary time, just from all the data that we have.
Things that we learned.
Number one is that being good
in one of these categories did not necessarily guarantee
being good at the others.
That's reflected in the data.
Two, is that Pixel 6A and the Realme 10 Pro+
turned out to be the best bang for the buck
As far as a smartphone camera from the phones
that came out in 2022.
And the Pixel 6A is the overall winner
matches with it winning our bracket-style
social media polling blind test from last year,
which the Pixel 5A won.
Last but not least,
Yeah, brighter photos do typically win,
but you can be too bright.
So I think this mission was a success. We learned a lot.
I'll say congrats to the Pixel 6A
on the People's Choice Award, so to speak.
And you know what?
We're gonna leave up the voting website till, let's say,
through the end of this year.
Till the end of January how about?
So January 31st is the last day you'll be able
to go to vote.mkbhd.com and blind-test yourself.
And I think we'll replace the letters now
with the actual names of the phones
Now that it's been revealed.
But we don't wanna have to pay
for all these terabytes of data processing forever.
So that's your deadline.
So thanks to y'all on Discord
for helping us beta test this thing.
Thanks to Zach for helping us build this site
and implement the ELO-style power ranking system.