Let $Alg\N$ be a nest algebra associated with the nest $\N$ on a (real or complex) Hilbert space $\mathcal{H}$. We say that $Alg\N$ is zero product determined if for every linear space $\V$ and every bilinear map $\phi:Alg\N \times Alg\N \rightarrow \V$ the following holds: if $\phi(A,B)=0$ whenever $AB=0$, then there exists a linear map $T$ such that $\phi(A,B)=T(AB)$ for all $A,B\in Alg\N$. If we replace in this definition the ordinary product by the Jordan (resp. Lie) product, then we say that $Alg\N$ is zero Jordan (resp. Lie) product determined. We show that any finite nest algebra over a complex Hilbert space is zero product determined, and it is also zero Jordan product determined. Moreover, we show that any finite-dimensional nest algebra on a (real or complex) Hilbert space is zero Lie (resp. associative, Jordan) product determined. In addition, we characterize separately strongly operator topology continuous bilinear map $\phi$ from $Alg\N \times Alg\N$ into a topological linear space $\V$ with the property that $\phi(A,B)=0$ whenever $AB=0$ or $\phi(A,B)=0$ whenever $AB+BA=0$.