What is "[core] language undefined behavior" as opposed to "undefined behavior"?

The slide you refer to mentions:

C++ already has great gobs of production UB-free code (*)

(*) core language UB in a context that requires a constant expression

The standard defines the term "undefined behaviour" and adds more details in a note 3.65:

3.65 [defns.undefined]

undefined behavior

behavior for which this document imposes no requirements

[Note 1: Undefined behavior may be expected when this document omits any explicit definition of behavior or when a program uses an incorrect construct or invalid data. Permissible undefined behavior ranges from ignoring the situation completely with unpredictable results, to behaving during translation or program execution in a documented manner characteristic of the environment (with or without the issuance of a diagnostic message ([defns.diagnostic])), to terminating a translation or execution (with the issuance of a diagnostic message). Many incorrect program constructs do not engender undefined behavior; they are required to be diagnosed. Evaluation of a constant expression ([expr.const]) never exhibits behavior explicitly specified as undefined in [intro] through [cpp]. — end note]

Note the last sentence. This is what they refer to when saying "production UB-free code" though it holds only for constant expressions and only behavior explicitly specified as undefined.

For example, you cannot dereference a null pointer (https://eel.is/c++draft/expr.unary.op#1). Outside of constexpr it is undefined. During evaluation of a constant expression the compiler has to diagnose it:

#include <array>

constexpr int foo() {
    int * x = 0; 
    return *x;
}

int main() {
    constexpr auto x = foo();
}

Error message from gcc:

<source>:3:15: error: constexpr function never produces a constant expression [-Winvalid-constexpr]
    3 | constexpr int foo() {
      |               ^~~
<source>:5:12: note: read of dereferenced null pointer is not allowed in a constant expression
    5 |     return *x;
      |            ^
<source>:9:20: error: constexpr variable 'x' must be initialized by a constant expression
    9 |     constexpr auto x = foo();
      |                    ^   ~~~~~
<source>:5:12: note: read of dereferenced null pointer is not allowed in a constant expression
    5 |     return *x;
      |            ^
<source>:9:24: note: in call to 'foo()'
    9 |     constexpr auto x = foo();
      |                        ^~~~~

If you remove constexpr from the code it's just undefined and there is no compiler error message from gcc (it crashes with a segfault, but it could do anything else).

Opposed to the cases where the standard explicitly specifies something as undefined, there are cases that are implicitly undefined just by the mere fact that the standard does not define them. It's not possible for compilers to diagnose all of that even in constexpr contexts.

Your understanding is mostly correct.

"Core language" means the language itself as opposed to its standard library.

UB in the core language should be always caught in constexpr contexts (per [expr.const]/10.8). As opposed to the UB in the library, which isn't caught in constexpr.

As noted in the other answer, the "implicit" UB also might not be caught, even in the core language (implicit = UB because the standard doesn't decribe the behavior, as opposed to explicitly banning it).

...as opposed to something like this

std::vector<int> v = getvec();
v[v.size()] = 3;

Where I'm doing out-of-bound access for a library entity like std::vector? (Well, at the end I'm still overflowing a an int[], though...)

^{(bold mine)}

Nope. Formally UB can't happen inside of a standard library function.

The spec for operator[] says that if you pass an out-of-bounds index, you get UB. You get UB immediately when you call it with a bad index, regardless of what it does internally.

What it does internally is an implementation detail, it's not subject to the same rules as the user code. Compilers may or may not be more lenient to what happens in the system headers.