I think I've figured out the answer to my question. Please take a look to see if it makes any sense (because it dosent really make any sense to me even though i figured it out, i suck at math).
I want only to check if the two rays intersect. I will go about it by calculating the direction of rotation of two "triangles" created from the two rays. They aren't really triangles but from a mathematical standpoint, if i only wanted to calculate the rotation of the triangle, i only need two vectors with a common starting point and the rest doesn't matter.
The first triangle will be formed by two vectors and a starting point. The starting point will be the first ray's starting point. The first vector will be the first ray's direction vector. The second vector will be the vector form the first ray's starting point to the second ray's starting point. From here we take the cross product of the two vectors and note the sign.
We do this again for the second triangle. Again, the starting point is the second ray's starting point. The first vector is the second ray's direction and the second vector is from the second ray's starting point to the first ray's starting point. We take the cross product again of the vectors and note the sign.
Now we simply take the two signs and check if they are the same. If they are the same, we have no intersection. If they are different we have an intersection. That's it!
Here's some psudo code:
sign1 = cross(vector1, point1 - point2)
sign2 = cross(vector2, point2 - point1)
if (sign1 * sign2 < 0) // if signs are mismatched, they will multiply to be negative
return intersection
works out to be 5 multiplications, 6 subtractions, and 1 comparison