Not necessarily. Every gene in our bodies will show a different pattern of inheritance, meaning that if you trace a single gene (or set of genes - i.e. the 'y' chromosome) you'll find a different point in our evolutionary past when those genes came into our modern genetic makeup. Some genes are very old - ribosomal RNAs, for example, have changed little since we separated from chimps. Others are far newer - i.e. the gene which allows some humans to consume milk as adults is only 8000-10,000 years old.
And, as Bill mentioned, it is not exactly clear where along the human evolutionary path truly separate species exist; it may be that the ancestors who incorperated this Y chromosome may be the same species as us (defined by being able to have viable offspring via interbreeding), or they may not.