Stingray v2.2

This release accompanies the review of Stingray from pyOpenSci. As such, it contains some internal changes that were driven by the review together with some new addition.

New Features

• Introduce FITSReader class for lazy-loading of event lists
  (#834)
• Add a compute_rms function to LombScarglePowerspectrum
  (#828)
• Implement the shift-and-add technique for QPOs and other
  varying power spectral features
  (#849)

Bug Fixes

• The fold_events function now checks if the keyword arguments
  kwargs are in the list of optional parameters. If
  any unidentified keys are present, it raises a
  ValueError. This fix ensures that the function only
  accepts valid optional parameters and provides a clear error message
  for unsupported keys.
  (#837)

Internal Changes

• Eliminated runtime dependency on setuptools
  (#852)
• Moved configuration to pyproject.toml as recommended by PEP 621
  (#842)
• Added pre-commit hooks in pre-commit-config.yaml
  (#847)
• Improved main page of the documentation
  (#748)

Release 2.1

v2.1 (2024-05-29)

New Features

• Add function to calibrate event lists based on RMF file (#804)
• Speed up computation of pds for large arrays (#808)
• Add support for XTE science event data (#816)
• A friendlier API for the non-paralyzable dead time model model (#800)

Bug Fixes

• Fix issue when setting a property from a FITS file read (#814)
• Fix case when analyze_segments has an invalid segment (#822)
• Substitute np.asarray with np.asanyarray everywhere, to avoid copying memory maps into memory if possible (#824)

Internal Changes

• Dead time model fixes: more stable computations, better plotting of check_A and check_B (#800)
• Bumped jinja version to 3.1.4 (#825)

Stingray Version 2.0

v2.0 (2024-03-13)

New Features

• Power colors à la Heil et al. 2015
  (#780)
• Calculate colors and intensities on a segment-by-segment basis in
  event lists
  (#781)
• A function to randomize data in small bad time intervals
  (#782)
• The Lomb Scargle Fourier Transform (fast and slow versions) and the
  corresponding LombScargleCrossspectrum{.interpreted-text
  role="class"} and LombScarglePowerspectrum{.interpreted-text
  role="class"}
  (#737)
• A JAX implementation of the Gaussian Process tool by Hubener et
  al for QPO detection and
  parameter analysis.
  (#739)
• Extend join operation for events to arbitrary array attributes, not
  just pi and energy
  (#742)
• Allow the creation of empty light curves.
  (#745)
• Make StingrayTimeseries into a generalized light curve, with a less
  strict naming but implementing much of the underlying computing
  useful for Lightcurve as well.
  (#754)
• Our fast implementation of histograms is now safer (failing safely
  to the equivalent numpy histogram functions), more consistent
  (ranges moved to range, for consistency with numpy), and accept
  complex weights as well!
  (#764)

Bug Fixes

• When rms is low, the calculation in compute_rms often gave
  [NaN]{.title-ref}. We now check for this situation and give 0 with
  an uncertainty as a result.
  (#736)
• Eliminates deprecated call to
  [enable_deprecations_as_warnings]{.title-ref}, and contextually,
  changes the code to be much more robust in catching harmful
  warnings.
  (#738)
• Changes Crossspectrum.plot() function to plot the actual real and
  imaginary parts instead of their absolute values.
  (#747)
• Make commits marked as [docs only] skip all CI but the docs build
  (#749)
• Update tstart and tseg when using Lightcurve.truncate()
  (#753)
• Changed list comprehension to generator expression to reduce memory
  usage.
  (#756)
• Fix a bug with segment sizes not exact multiples of dt when dealing
  with light curves
  (#760)
• Fix a bug when light curve segments contain complex values
  (#760)
• Crossspectrum had "real" as default value. This meant that, for
  example, lags could not be calculated. Now the default value is
  "all", as it should be.
  (#762)
• Fix plotting of spectra, avoiding the plot of imaginary parts of
  real numbers
  (#763)
• Various bugfixes in [gti.py]{.title-ref}, and a new function to
  interpret the mix of multiple GTIs.
  (#774)
• Fixed subcs duplication by adding a check in the for loop that
  copies the attributes from table's meta items.
  (#776)
• Various bug fixes in DynamicalPowerspectrum, on event loading and
  time rebinning
  (#779)
• Fix issue with the Poisson noise calculation in lag spectra, that
  produced NaN errors under some conditions
  (#789)
• Fix rms computation and error bars
  (#792)
• Fix issue with Powerspectrum of a single light curve
  (#663)
• Fix nphots estimate in accelsearch, that lead to an underestimation
  of the power of candidates
  (#807)

Breaking Changes

• Eliminate deprecated format_ keyword from read and write methods.
  (#729)
• Remove legacy interface and obsolete large data machinery.
  (#755)
• Eliminate deprecated white_noise_level keyword from compute_rms.
  (#792)

Internal Changes

• Speedup creation of events in EventList.from_lc
  (#757)
• Separate slow tests from quick ones
  (#758)
• Use Readthedocs for documentation building
  (#769)
• More informative GTI messages
  (#787)
• Eliminated the usage of astropy logging
  (#799)

Stingray 2.0.0, release candidate 1

New Features

• Power colors à la Heil et al. 2015
  (#780)
• Calculate colors and intensities on a segment-by-segment basis in
  event lists
  (#781)
• A function to randomize data in small bad time intervals
  (#782)
• The Lomb Scargle Fourier Transform (fast and slow versions) and the
  corresponding LombScargleCrossspectrum{.interpreted-text
  role="class"} and LombScarglePowerspectrum{.interpreted-text
  role="class"}
  (#737)
• A JAX implementation of the Gaussian Process tool by Hubener et
  al for QPO detection and
  parameter analysis.
  (#739)
• Extend join operation for events to arbitrary array attributes, not
  just pi and energy
  (#742)
• Allow the creation of empty light curves.
  (#745)
• Make StingrayTimeseries into a generalized light curve, with a less
  strict naming but implementing much of the underlying computing
  useful for Lightcurve as well.
  (#754)
• Our fast implementation of histograms is now safer (failing safely
  to the equivalent numpy histogram functions), more consistent
  (ranges moved to range, for consistency with numpy), and accept
  complex weights as well!
  (#764)

Bug Fixes

• When rms is low, the calculation in compute_rms often gave
  [NaN]{.title-ref}. We now check for this situation and give 0 with
  an uncertainty as a result.
  (#736)
• Eliminates deprecated call to
  [enable_deprecations_as_warnings]{.title-ref}, and contextually,
  changes the code to be much more robust in catching harmful
  warnings.
  (#738)
• Changes Crossspectrum.plot() function to plot the actual real and
  imaginary parts instead of their absolute values.
  (#747)
• Make commits marked as [docs only] skip all CI but the docs build
  (#749)
• Update tstart and tseg when using Lightcurve.truncate()
  (#753)
• Changed list comprehension to generator expression to reduce memory
  usage.
  (#756)
• Fix a bug with segment sizes not exact multiples of dt when dealing
  with light curves
  (#760)
• Fix a bug when light curve segments contain complex values
  (#760)
• Crossspectrum had "real" as default value. This meant that, for
  example, lags could not be calculated. Now the default value is
  "all", as it should be.
  (#762)
• Fix plotting of spectra, avoiding the plot of imaginary parts of
  real numbers
  (#763)
• Various bugfixes in [gti.py]{.title-ref}, and a new function to
  interpret the mix of multiple GTIs.
  (#774)
• Fixed subcs duplication by adding a check in the for loop that
  copies the attributes from table's meta items.
  (#776)
• Various bug fixes in DynamicalPowerspectrum, on event loading and
  time rebinning
  (#779)
• Fix issue with the Poisson noise calculation in lag spectra, that
  produced NaN errors under some conditions
  (#789)
• Fix rms computation and error bars
  (#792)

Breaking Changes

• Eliminate deprecated format_ keyword from read and write methods.
  (#729)
• Remove legacy interface and obsolete large data machinery.
  (#755)
• Eliminate deprecated white_noise_level keyword from compute_rms.
  (#792)

Internal Changes

• Speedup creation of events in EventList.from_lc
  (#757)
• Separate slow tests from quick ones
  (#758)
• Use Readthedocs for documentation building
  (#769)
• More informative GTI messages
  (#787)

Version 1.1.2

New Features

• Phase Dispersion Minimization as a method to search for periodic
  signals in data is now implemented in the stingray.pulse
  submodule. To use it, you can use the phase_dispersion_search
  function in stingray.pulse.search. The accompanying statistical
  tests are located in the stingray.stats module, under
  phase_dispersion_probability, phase_dispersion_logprobability
  and phase_dispersion_detection_level.
  (#716)
• Add is_sorted function, to test if an array is sorted.
  (#723)
• Check if invalid data are inside GTIs, and warn or raise exception
  accordingly
  (#730)

Bug Fixes

• The method apply_gtis of the class Lightcurve is applied to all the
  attributes of the class Lightcurve. This works for both inplace=True
  and inplace=False
  (#712)
• Avoid allocation of an unneeded square matrix to improve memory
  management in _als (fix Issue 724)
  (#725)
• Fix Issue #726 -- Loading events without fmt keyword crashes
  (#727)

Documentation

• Reordered information about contributions with new black and
  towncrier procedures
  (#721)

Internal Changes

• Using towncrier to generate
  the changelogs.
  (#697)
• Added stingray's logo in the documentation's favicon and top bar.
  (#707)
• Improved contributing workflow by appending black codestyle
  configuration to pyproject.toml and ignoring PEP-8 non-compliant
  E203, W503 in flake8.
  (#715)
• Added a scrollbar to sidebarwrapper
  (#718)
• Simplify numba mocking code, and possibly improve code coverage
  estimate
  (#731)

v1.1

v1.1 (2022-10-02)

Bug fixes

• IMPORTANT: Fixed sign of time lags, which were calculated using the interest band as the reference.
• Fixed an issue when the fractional exposure in FITS light curves is slightly >1 (as sometimes happens in NICER data)

New

• Implemented the bexvar variability estimation method (Bayesian excess variance) for light curves.

Improvements

• A less confusing default value of segment_size in Z searches

Version 1.0

TL,DR:
These things will break your code with v1.0:

• Python version < 3.8
• The gtis keyword in pulse/pulsar.py (it is now gti, without the 's')

New

• Dropped support to Python < 3.8
• Multi-taper periodogram, including a Lomb-Scargle implementation for non-uniformly sampled data
• Create Count rate spectrum when calculating spectral-timing products
• Make modulation upper limit in (Averaged)Powerspectrum work with any normalization (internally converts to Leahy for the calculation)
• Implement Gardner-Done normalization (1 for perfect correlation, -1 for perfect anticorrelation) for Auto/Crosscorrelation
• New infrastructure for converting EventList and LightCurve objects into Astropy TimeSeries
• New infrastructure for converting most Stingray classes into Astropy Table objects
• Save and load of most Stingray classes to/from many different file formats (pickle, ECSV, HDF5, FITS, and all formats compatible with Astropy Table)
• Accept input EventList in DynamicalPowerSpectrum
• New stingray.fourier module containing the basic timing products, usable on numpy arrays, and centralizes fft import
• New methods in Crossspectrum and Powerspectrum to load data from specific inputs: from_events, from_lightcurve, from_time_array, from_lc_list (from_time_array was also tested using memory-mapped event lists as inputs: useful in very large datasets)
• New and improved spectral timing methods: ComplexCovarianceSpectrum, CovarianceSpectrum, LagSpectrum, RmsSpectrum
• Some depreciated features are now removed
• PSDLogLikelihood now also works with a log-rebinned PDS

Improvements

• Performance on large data sets is VASTLY improved
• Lots of performance improvements in the AveragedCrossspectrum and AveragedPowerspectrum classes
• Standardized use of new fast psd/cs algorithm, with legacy still available as an alternative option to specify
• Reading calibrated photon energy from event files by default
• In pulse/pulsar.py, methods use the keyword gti instead of gtis (for consistency with the rest of Stingray)
• Added more documentation about using GTIs for a Powerspectrum or Crossspectrum object if you already passed a GTI to the input Lightcurve object
• Moved CovarianceSpectrum to VarEnergySpectrum and reuse part of the machinery
• Improved error bars on cross-spectral and spectral timing methods
• Measure absolute rms in RmsEnergySpectrum
• Friendlier pyfftw warnings
• PDS/CrossSp initially store the unnormalized power, and convert it on the fly when requested, to any normalization
• Adjust codecov settings with new yaml file

Bug fixes

• Fixed error bars and error distributions for sliced (iterated) light curves and power spectra
• Fixed a bug in how the start time when applying GTIs (now using the minimum value of the GTI array, instead of half a time bin below the minimum value)
• All simulator errors are zero
• Fixed coherence uncertainty
• Documented a Windows-specific issue when large count rate light curves are defined as integer arrays (Windows users should use float or specify int-64)
• If the variance of the lightcurve is zero, the code will fail to implement Leahy normalization
• The value of the PLEPHEM header keyword is forced to be a string, in the rare cases that it's a number
• and more!

Full Changelog: v0.3...v1.0

Version 1.0 beta

What's Changed

TL,DR: these things will break your code with v1.0beta:

• Python version < 3.8
• The gtis keyword in pulse/pulsar.py (it is now gti, without the 's')

New

• Dropped support to Python < 3.8
• Multi-taper periodogram, including a Lomb-Scargle implementation for non-uniformly sampled data
• Create Count rate spectrum when calculating spectral-timing products
• Make modulation upper limit in (Averaged)Powerspectrum work with any normalization (internally converts to Leahy for the calculation)
• Implement Gardner-Done normalization (1 for perfect correlation, -1 for perfect anticorrelation) for Auto/Crosscorrelation
• New infrastructure for converting EventList and LightCurve objects into Astropy TimeSeries
• New infrastructure for converting most Stingray classes into Astropy Table objects
• Save and load of most Stingray classes to/from many different file formats (pickle, ECSV, HDF5, FITS, and all formats compatible with Astropy Table)
• Accept input EventList in DynamicalPowerSpectrum
• New stingray.fourier module containing the basic timing products, usable on numpy arrays, and centralizes fft import
• New methods in Crossspectrum and Powerspectrum to load data from specific inputs: from_events, from_lightcurve, from_time_array, from_lc_list (from_time_array was also tested using memory-mapped event lists as inputs: useful in very large datasets)
• New and improved spectral timing methods: ComplexCovarianceSpectrum, CovarianceSpectrum, LagSpectrum, RmsSpectrum
• Some depreciated features are now removed
• PSDLogLikelihood now also works with a log-rebinned PDS

Improvements

• Performance on large data sets is VASTLY improved
• Lots of performance improvements in the AveragedCrossspectrum and AveragedPowerspectrum classes
• Standardized use of new fast psd/cs algorithm, with legacy still available as an alternative option to specify
• Reading calibrated photon energy from event files by default
• In pulse/pulsar.py, methods use the keyword gti instead of gtis (for consistency with the rest of Stingray)
• Moved CovarianceSpectrum` to VarEnergySpectrum`` and reuse part of the machinery
• Improved error bars on cross-spectral and spectral timing methods
• Measure absolute rms in RmsEnergySpectrum
• Friendlier pyfftw warnings
• Streamline PDS/CrossSp production, adding from_events, from_lc, from_lc_iterable, and from_time_array (to input a numpy array) methods
• PDS/CrossSp initially store the unnormalized power, and convert it on the fly when requested, to any normalization
• Adjust codecov settings with new yaml file

Bug fixes

• Fixed error bars and err_dist for sliced (iterated) light curves and power spectra
• Fixed a bug in how the start time when applying GTIs (now using the minimum value of the GTI array, instead of half a time bin below the minimum value)
• All simulator errors are zero
• Fixed coherence uncertainty
• Documented a Windows-specific issue when large count rate light curves are defined as integer arrays (Windows users should use float or specify int-64)
• If the variance of the lightcurve is zero, the code will fail to implement Leahy normalization
• The value of the PLEPHEM header keyword is forced to be a string, in the rare cases that it's a number
• and more!

Full Changelog: v0.3...v1.0-beta

Version 0.3

New features and changes:

• Lots of performance improvements
• Faster simulations
• Averaged Power spectra and Cross spectra now handle Gaussian light curves correctly
• Fixes in rebin functions
• New statistical functions for signal detection in power spectra and pulsar search periodograms
• Much improved FTOOL-compatible mission support
• New implementation of the FFTFIT method to calculate pulsar times of arrival
• H-test for pulsar searches
• Z^2_n search adapted to binned and normally distribute pulse profiles
• Large data processing (e.g. from NICER) allowed
• Rebinning function now accepts unevenly sampled data
• New saving and loading from/to Astropy Tables and Timeseries
• Improved I/O to ascii, hdf5 and other formats
• Rehaul of documentation, now on stingray.science

Version 0.2

Performance improvements:

• Much faster loading of light curves

• Faster testing

• Faster pulsar searches

New functionalities:

• Fourier-domain accelerated pulsar searches

• Fourier Amplitude Difference correction for power density spectra affected by dead time

Infrastructure changes:

• Adaptation of package to APE17

Major bug fixes:

• Rms Normalization of PDSs was wrong under certain conditions

• A test for numpy complex types done wrong produced lag measurements identical to 0

Plus many, many smaller bug fixes and performance tweaks. Enjoy!