You need to self join the table to get the records you require.
For this answer I created your table without the foreign keys as they are not required to get it to work.
CREATE TABLE Rankings (
id int IDENTITY NOT NULL,
user_id INT NOT NULL,
game_poule_id INT NOT NULL,
rank INT NOT NULL,
insertDate DATETIME NOT NULL
);
Insert some sample data. Without more information I cannot simulate any better than this.
INSERT Rankings(user_id,game_poule_id,rank,insertDate)
VALUES(1, 100, 3, CURRENT_TIMESTAMP-2)
INSERT Rankings(user_id,game_poule_id,rank,insertDate)
VALUES(1, 100, 2, CURRENT_TIMESTAMP-1)
INSERT Rankings(user_id,game_poule_id,rank,insertDate)
VALUES(1, 101, 6, CURRENT_TIMESTAMP)
INSERT Rankings(user_id,game_poule_id,rank,insertDate)
VALUES(2, 100, 5, CURRENT_TIMESTAMP-2)
INSERT Rankings(user_id,game_poule_id,rank,insertDate)
VALUES(2, 100, 1, CURRENT_TIMESTAMP-1)
INSERT Rankings(user_id,game_poule_id,rank,insertDate)
VALUES(2, 101, 2, CURRENT_TIMESTAMP)
Query for the last but one rank
SELECT Rankings.game_poule_id, Rankings.user_id, rank, MAX(Rankings.insertDate)
FROM Rankings INNER JOIN
(SELECT game_poule_id, user_id, MAX(insertDate) max_insert_date
FROM rankings
GROUP BY game_poule_id, user_id) Max_Ranking_Date
ON Rankings.user_id = Max_Ranking_Date.user_id
AND Rankings.insertDate < Max_Ranking_Date.max_insert_date
AND Rankings.game_poule_id = Max_Ranking_Date.game_poule_id
GROUP BY Rankings.game_poule_id, Rankings.user_id, rank
PLEASE NOTE!
As you can see from the results you will not get a ranking for a game that only has one row per user. But since you are asking for the "last but one" that only makes sense for games with multiple entries.
EDIT:
I've just realised the query I have provided will not return one row per user per game. Anyone want to fix it? I have to get on with some work :)