Nautilus is a massive tank of a champion who brings survivablilty and CC to a team. At all stages of the game, Nautilus has higher than average survivability and a lot of Crowd Control to disrupt teamfights with, most often targeting priority backliners when it comes to group engagements, and trying to scatter the enemy team to make the carries easier targets to eliminate
Passive – Staggering Blow
Nautilus's basic attacks deal bonus physical damage and root his target
Q – Dredge Line
Nautilus hurls his anchor forward in a line. Enemies hit are Airborne icon dragged towards Nautilus and he simultaneously dashes to them, dealing them magic damage and briefly stunning them
W – Titans Wrath
Nautilus shields himself for up to 10 seconds, causing his basic attacks to deal bonus magic damage over 2 seconds to his target and all enemies around them while the shield holds.Titan's Wrath resets Nautilus' autoattack timer
E – Riptide
Nautilus creates 3 waves of explosions that radiate from him, each dealing magic damage to all enemies hit and Slow icon slowing them by an amount decaying over 1.5 seconds.Enemies can be hit by multiple waves, with each wave beyond the first dealing half damage.
R – Depth Charge
Nautilus sends out a depth charge that chases a Depth Charge Target target enemy champion, dealing magic damage, Airborne icon knocking them up for 1 second and stunning them on arrival. All enemies hit along the way are also dealt magic damage, are knocked up for 1 second, and are also stunned.
A lot of Nautilus' trading potential is based on his Titans Wrath, which has an 18 second cooldown. It's significantly easier to outdamage him and gain dominant positioning in lane, and even freeze the lane in some cases given how few ways he has to unfreeze it, with no long range abilities to do so, without putting himself in harms way.
Putting minions in between you and Nautilus can win you the lane. He can't combo you if he misses his Dredge Line, and it'll collide with anything it touches, pulling him towards it and his damage is negligible if he misses an ability, without sunfire cape, meaning that early trades can be yours to take
Run away from Depth Charge. The ability has more CC than damage, and once it's over Nautilus doesn't have access to it thanks to it's cooldown being consistently longer than a minute, and as you reach the mid game, you'll be able to punish him for not having it. It also makes him harder to gank for.
She just wins this lane. Outdamages him in trades, and has the ability to stun him for every trade he takes by riposting his first auto attack. She also has the mobility to dodge out of his Dredge Line if she predicts it's timing correctly (It'll typically be right after he gets free of the riposte stun) as well as the ability to kill him consistently due to just how hard she beats him, and to snowball really, really hard as a result and carry the entire game herself.
Riven has the damage to punish Nautilus' long cooldowns, as pieces of her combo rotation will be up twice before his Titans Wrath comes back up once. Riven also has an AD scaling shield to mitigate enough of Nautilus damage to outtrade him, the mobility to chase him down, or run away from the Depth Charge, the CC to stop him from running or to stop him from chasing and the ability to snowball from her leads and carry the game.
He can't kill Nasus, he just doesn't have the damage to compete with Nasus lifesteal, especially without ignite. But if he takes ignite, he'll get beaten for different reasons. Nasus can just stack as long as he has the vision to ensure that he won't die to a gank, and then at 6 he becomes even less able to die. If he's been farming, he can even kill Nautilus, and naturally, Nasus can and will carry the game.
The ability to run from the Depth Charge? Gnar has that. The ability to kill Nautilus as time goes on? Gnar has that. The ability to become a highly damaging pseudo-tank that carries the game off of solo killing Nautilus? Gnar has that too. Gnar is just generally good, and when he gets his hands on a Frozen Mallet he has serious kill pressure on a lone Nautilus and as a result, splitpush pressure too, since the top laner can no longer stop him.
Aside from the early game, Nautilus doesn't do much damage without building AP, so you can generally just focus on building your own core items. If you find yourself taking too much damage from early trades, you can buy some early MR, but make sure you check the minion wave sizes to ensure that's not where all your health is going.