When using WooCommerce with the Astra theme installed, you can customize the Cart icon. You can select the cart icon of your choice from the available options in the customizer settings.
Navigate to Appearance > Customize > Header builder > Cart. Here, you can select the cart icon from the available 3 options.
In case you want to add the CSS Astra cart icons, you need to add the following custom code to the functions.php file.
add_filter( 'astra_woocommerce_cart_icon', '__return_false' );
After adding the above filter, the cart icons customizing options would look like this.
Note:
For additional details on how to add a custom code to the functions.php file please see this article: How to Add Custom PHP Code?