The dynamic programming approach you describe is called DAG shortest path. It only works on directed acyclic graphs (that is, graphs with no cycles). Its asymptotic runtime is O(V+E) (where V and E are the number of vertices and edges, respectively), which is faster than Dijkstra's algorithm.
I'm not sure, but are you asking how to calculate the number of paths which has length equal to the shortest path?
You can do that by maintaining predecessor information when you calculate the length of the shortest path. When you move to node j, store all pairs (i,j) such that going from i to j is part of a shortest path. One way to implement this is to add two fields, D(x,y).optimalUp and D(x,y).optimalRight (datatype boolean), indicating if you get an optimal solution if you entered (x,y) by going up or right, respectively. For example, set D(x,y).optimalUp to true if going up from (x,y-1) results in the cheapest path.
Then you can do a second pass to count the number of cheapest paths, using dynamic programming. Add another field, say D(x,y).count (integer) which holds the number of ways to go from A to (x,y) in the cheapest way. There are two ways to reach (x,y): Via (x-1,y) and (x,y-1). The count of (x,y-1) should only be added to the count of (x,y) if it is possible to achieve the cheapest path to (x,y) by going via (x,y-1). The same principle holds for (x-1,y).
We then get the recurrence:
D(x,y).count =
D(x,y).optimalUp ? D(x,y-1).count : 0
+ D(x,y).optimalDown ? D(x-1,y).count : 0
( ?: is the conditional operator in C/C++/Java.)
Judging from your picture, it seems your graph is a grid. Beware that going only up or right need not result in the shortest path. In the graph below, the shortest path from A to B is 8, but you cannot achieve better than 12 if you go only right and up.
x -1-- x -1-- B
| | |
1 9 9
| | |
x -1-- x -1-- x
| | |
9 9 1
| | |
A -1-- x -1-- x
Since this is marked as homework, I won't be providing more details than this for now. This should nevertheless be a nudge in the right direction (if I have understood your problem correctly). Feel free to ask follow-up questions, though.