He beat his compatriot Gianni Pittella into second place by 287 votes to 200.
But it was not enough to win outright and so voting for third round is underway.
Tajiani is a member of the centre-right EPP group, while Pittella sits with the centre-left Socialists and Democrats.
Liberal MEP Guy Verhofstadt pulled out of the race to throw his weight behind Tajiani, an ex-spokesman for former Italian PM Silvio Berlusconi, who is now favourite to win.
It means his EPP grouping will control all three top jobs at the EU level: the European Parliament, the European Commission, and the European Council.
This could see Socialists ask for a reshuffle to secure at least one position.
Voting takes place by secret ballot; an absolute majority is needed to win.
If after three rounds of voting, such a majority is not secured then a fourth round will be held.
A simple majority is all that is required in that final round. If there is still a tie, the oldest candidate takes the top job.