RL-OptS

Analytical functions

  • Get started
  • Documentation
    • RL framework
      • Classic version
      • numba implementation
        • Reinforcement learning environments
        • Reinforcement learning agents
    • Learning and benchmarking
    • Imitation learning
    • Analytical functions
    • Utils
  • Tutorials
    • Reinforcement Learning
    • Benchmarks
    • Imitation learning
    • Learning to reset in target search problems

On this page

  • Step length distribution generators
    • pdf_multimode
    • pdf_powerlaw
    • pdf_discrete_sample
  • Calculation of policy from step length PDF
    • get_policy_from_dist

Report an issue

Analytical functions

Step length distribution generators


source

pdf_multimode

 pdf_multimode (L:int, lambdas:list, probs:list)

Computes the discrete PDF of multi-mode exponential of the form

\[ \Pr(L) = \sum_{i=1,2} \omega_i (1-e^{-1/\lambda_i}) e^{-(L-1)/\lambda_i} \, , \] where \(\omega\) is the probability of each mode and \(\lambda\) it’s scale.

Type Details
L int Either int or array for which pdf is calculated
lambdas list Scales of each modes
probs list Probability weight of each mode
Returns array Array with probability of each L
lambdas = np.array([2,15])
probs = np.array([0.99, 0.01])
L_max = 100
plt.loglog(np.arange(1, L_max),
           pdf_multimode(L = np.arange(1, L_max), lambdas=lambdas, probs=probs)
          )
plt.xlabel('L'); plt.ylabel('P(L)')
Text(0, 0.5, 'P(L)')


source

pdf_powerlaw

 pdf_powerlaw (L:float, beta:float=1)

Computes the discrete PDF of a powerlaw of the form \[ \Pr(L)\sim L^{-1-\mathrm{beta}}. \]

Type Default Details
L float Either int or array for which pdf is calculated
beta float 1 Exponent of the power law
Returns array Array with probability of each L
plt.loglog(np.arange(1, 1000),
           pdf_powerlaw(L = np.arange(1,1000), beta = 1)
          )
plt.xlabel('L'); plt.ylabel('P(L)')
Text(0, 0.5, 'P(L)')


source

pdf_discrete_sample

 pdf_discrete_sample (pdf_func:object, num_samples:int, **args_func)

Samples discrete values from a given PDF

Type Details
pdf_func object Function generating the pdf
num_samples int Number of samples to create
args_func
Returns array Samples
samples = pdf_discrete_sample(pdf_func = pdf_multimode, 
                              num_samples=10000, 
                              L = np.arange(1, L_max), lambdas=lambdas, probs=probs)

counts = np.bincount(samples)[1:]

plt.loglog(np.arange(1, len(counts)+1),
           counts/counts.sum(), 'o', 
           label = 'Histogram samples')

plt.loglog(np.arange(1, L_max),
           pdf_multimode(L = np.arange(1, L_max), lambdas=lambdas, probs=probs),
           label = 'Theory'
          )
plt.xlabel('L'); plt.ylabel('P(L)'); plt.legend()
<matplotlib.legend.Legend>

Calculation of policy from step length PDF


source

get_policy_from_dist

 get_policy_from_dist (n_max, func, renorm=True, **args_func)

Given a PDF of step lengths, calculates the corresponding policy

Type Default Details
n_max Maximum counter n_max for which the policy is calculated
func Function generating the pdf
renorm bool True If true, we check whether the distribution has a boundary N, for which _n=N^Pr(L=nd) = 0
args_func
Returns array Policy at each counter value
L = 100
betas = np.linspace(0.1, 2, 10)
colors = plt.cm.plasma(np.linspace(0,1,len(betas)+2))

fig, ax = plt.subplots()
for beta, color in zip(betas, colors):
    
    policy = get_policy_from_dist(n_max = L, 
                                  func = pdf_powerlaw,
                                  beta = beta)
    ax.plot(np.arange(2, L+1), policy[1:], c = color)


# Plot features    
plt.setp(ax, xlabel =r'$n$', ylabel = r'$\pi(\uparrow|n)$', xscale = 'log')
cbar = fig.colorbar(plt.cm.ScalarMappable(norm= mcolors.Normalize(vmin=betas.min(), 
                                                                  vmax=betas.max()),
                                          cmap=plt.cm.plasma),
                    ax = ax)
cbar.set_label(r'$\beta$')