This work directly bakes the safety and task constraints into the boundary conditions of the second-order Hamilton-Jacobi Belllman (HJB) Equation. As a result, the solution to this HJB equation is a safe value function that sharpens the distinction between safe and unsafe states and can guide the policy to achieve goals safely. By treating safety and task as boundary conditions, we move from complex, dense rewards to more straightforward, sparse constraints. Additionally, we also propose a hybrid model that combines a mesh-based function approximator for accurately computing boundary conditions with a meshless method, such as neural networks or kernel functions, to enhance computational efficiency. This work is accepted by the International Journal of Robotics Research (IJRR) [Paper].