Not necessarily. You can only output so much force.
F = m*a for force, mass, and acceleration
K=m*v^2 for kinetic energy, mass, and velocity
Assuming we start from rest: v = a*t, for velocity, acceleration, and time.
Thus, v = (F/m)*t
So K = m*((F/m)*t)^2
Which simplifies to: K = ((F*t)^2)/m
Since the force you output is the same regardless of what weapon you're using (a bigger sword does not magically make you stronger IRL), and since you will only ever have so much time to prepare a blow before the other guy shanks you in the ribs, it's safe to say that F and t are constant; meaning that K is inversely proportional to m.
In other words, the heavier your weapon, the less kinetic energy you can impart on the target with a swing. Further, bigger surface area means worse penetration, so you might very lightly bruise his entire chest while he puts a small hole right through your lung.
Now, there are some real-world considerations that mean the tiniest sword possible is not always the best. First of all, your arm has mass, and causes drag. Drag is proportional to velocity and drag coefficient, and your arm is not optimised to minimise drag. So you want to keep velocity to an acceptable level while swinging your sword. Secondly, material properties. A weapon strong enough to withstand the demands of hacking and stabbing at bits of steel all day is going to need a certain level of structural integrity and that level of structural integrity is going to require a certain amount of material.
This leads us to the weapons of war that we see in real life: minimal surface area on impact point (seriously, look up what a real-life warhammer looks like. It has more in common with a workman's claw hammer than the fantasy brick of steel on the end of a pole), just about as light as possible, thin-bladed, and with a focus on point-impacts rather than cutting (war-axes were incredibly uncommon and things like halberds were designed for hooking and spiking, not cutting).