There is a general definition of the Euler characteristic of a chain complex; one way to say it is the following. If $0 \to V_0 \to V_1 \to \dots \to V_n \to 0$ is a chain complex of finite-dimensional vector spaces then its Euler characteristic can be defined as the alternating sum of dimensions
$$\chi(V) = \sum_{i=0}^n (-1)^i \dim V_i$$
and if the $V_i$ are furthermore graded then we can take the graded dimension instead, etc. The general fact, which is a kind of "inclusion-exclusion" argument, is that the Euler characteristic is also given by the alternating sum of dimensions
$$\chi(V) = \sum_{i=0}^n (-1)^i \dim H_i(V)$$
of the homology of $V$. Roughly speaking, $\chi(V)$ is a homological version of "the dimension of $V$," and is in some sense the most basic invariant of $V$.
The connection to the Euler characteristic of a topological space $X$ is that we can take $V$ to be any of several different chain complexes whose homology computes the homology of $X$. In general we can take singular chains, which is infinite-dimensional, so we can't discuss the alternating sum of dimensions and can only discuss the alternating sum of homologies. But, for example, if $X$ is a finite CW complex then we can take $V$ to be cellular chains, which is finite-dimensional, of dimension $c_i$ in degree $i$ where $c_i$ is the number of $i$-cells. This implies that the Euler characteristic of $X$ can be computed as an alternating sum over the number of cells,
$$\chi(X) = \sum_{i=0}^n (-1)^i c_i$$
which is a generalization of Euler's polyhedral formula; the fact that this is also the alternating sum of the homologies implies that the alternating sum of the number of cells is a homotopy invariant. Roughly speaking $\chi(X)$ is a homotopy-theoretic version of cardinality, and is also in some sense the most basic invariant of $X$ (from the point of view of homotopy theory).
The basic connection to the Hilbert function is ultimately not that interesting as far as I can tell, it just boils down to the definition of a resolution. A free resolution is a chain complex of free modules $F$ such that $H_0(F) \cong M$ and the higher homologies vanish. This means its Euler characteristic is $\chi(F) = \dim H_0(F) = \dim M$ and the same is true for graded dimensions, etc. This is again a kind of "inclusion-exclusion" argument, and it's a useful computational tool but it doesn't by itself imply any kind of deep relationship to the topological Euler characteristic, which involves some other unrelated chain complex, with interesting homologies in higher degree but no extra grading.
However there is a different connection to the Hilbert function which is less direct, and which passes through the Hirzebruch-Riemann-Roch theorem. Namely, the Hilbert function of, say, a smooth projective variety $X \subset \mathbb{CP}^n$ is eventually equal to a polynomial, the Hilbert polynomial $HP(d)$. The HRR theorem implies that the Hilbert polynomial is given by a sum
$$HP(d) = \chi(X, \mathcal{O}(d)) = \int_X \text{ch}(\mathcal{O}(d)) \text{td}(X)$$
where
- $\chi$ here is now the Euler characteristic of a coherent sheaf, which is defined as the alternating sum over the dimensions of its sheaf cohomology; for large $d$ the higher cohomology vanishes (by Serre vanishing) and this is the same as $\dim H^0(X, \mathcal{O}(d))$, which is the dimension of homogeneous polynomials on $X$ of degree $d$, and this is why we get the Hilbert polynomial.
- $\mathcal{O}(d)$ is the line bundle on $\mathbb{P}^n$ whose sections are the homogeneous polynomials of degree $d$, pulled back to $X$.
- $\text{ch}$ and $\text{td}$ are the Chern character and Todd class, and
- $\int_X$ refers to pairing a cohomology class with the fundamental class of $X$.
This means the Hilbert polynomial contains information about the Chern classes of $X$. And one of these Chern classes, namely the top Chern class $c_{n-1}$, does in fact know the topological Euler characteristic of $X$! However, this information is mixed up with information about the Chern classes of $\mathcal{O}(d)$, so it's not as simple as saying that the Hilbert polynomial computes $\chi(X)$. The Todd class also mixes things up a lot, so I don't even know if we get that the Hilbert polynomial determines $\chi(X)$ in general.
That was a lot of abstraction so let's get more concrete. Suppose $X \subset \mathbb{CP}^2$ is a smooth projective plane curve over $\mathbb{C}$, such as an elliptic curve in Weierstrass form. In this case HRR specializes to the Riemann-Roch theorem, and we get that the Hilbert polynomial of $X$ is
$$HP(d) = d - g + 1.$$
So the Hilbert polynomial knows the genus $g$ of $X$, which is the same information as $\chi(X) = 2 - 2g$. But they're not the same number, and the degree $d$ is in there too (which is not an invariant of $X$ and depends on the embedding into projective space). Note, interestingly, that the constant term
$$HP(0) = \chi(X, \mathcal{O}_X) = 1 - g$$
of the Hilbert polynomial is exactly half $\chi(X)$; here $\chi(X, \mathcal{O}_X)$ is not $\chi(X)$ but the "coherent Euler characteristic"
$$\chi(X, \mathcal{O}_X) = \sum (-1)^i \dim H^i(X, \mathcal{O}_X).$$
This is also not the value $H(0)$ of the Hilbert function at $0$; that would just be $\dim H^0(X, \mathcal{O}_X) = 1$. This does not directly generalize to higher dimensions but it's not a coincidence and can be explained using Hodge theory; for some more details see the fifth proof here.
