Sequential decision making algorithms are history dependent policies. Modern sequence prediction models such as transformer architectures have made it feasible to successfully represent these objects in compact architectures. In these research works we explore how to meta-train these models from data to encode known and new sequential decision making strategies.