Hippocampal Reactivation Extends for Several Hours Following Novel Experience

This Accepted Manuscript has not been copyedited and formatted. The final version may differ from this version. A link to any extended data will be provided when the final version is posted online.

Research Articles: Systems/Circuits

Hippocampal reactivation extends for several hours following novel experience

Bapun Giri1,2, Hiroyuki Miyawaki1,3, Kenji Mizuseki3,4, Sen Cheng5 and Kamran Diba1,2

1Dept of Psychology, University of Wisconsin-Milwaukee, PO Box 413, Milwaukee, WI 53201 2Dept of Anesthesiology, 1150 W Medical Center Dr, University of Michigan, Ann Arbor, MI 48109 3Dept of Physiology, Osaka City University Grad School of Medicine, Asahimachi 1-4-3, Abeno-ku, Osaka, 545-8585, JAPAN 4Center for Molecular and Behavioral Neuroscience, Rutgers University, Newark, NJ 07102 5Institut fur Neuroinformatik, Ruhr-University Bochum, Bochum, Germany https://doi.org/10.1523/JNEUROSCI.1950-18.2018

Received: 23 July 2018

Revised: 13 November 2018

Accepted: 26 November 2018

Published: 10 December 2018

Author contributions: B.G., H.M., S.C., and K.D. designed research; B.G., H.M., and K.M. performed research; B.G. and K.D. analyzed data; B.G., H.M., S.C., K.M., and K.D. edited the paper; B.G. and K.D. wrote the paper; K.D. wrote the first draft of the paper.

Conflict of Interest: The authors declare no competing financial interests.

We are grateful to Asohan Amarasingham, Laura Ball, Caleb Kemere, Andrew P. Maurer, and Brendon Watson for helpful comments, and to Andres Grosmark, John Long and Gyorgy Buzsaki for publicly sharing their valuable data. This project was funded by the National Institute of Mental Health (NIMH) R01MH109170 and R01MH117964 to K.D., with additional support to S.C. from the German BMBF 01GQ1506 and to K.M. from the Toray Science Foundation.

Correspondence should be addressed to KD: [email protected]

Cite as: J. Neurosci 2018; 10.1523/JNEUROSCI.1950-18.2018

Alerts: Sign up at www.jneurosci.org/alerts to receive customized email alerts when the fully formatted version of this article is published.

Accepted manuscripts are peer-reviewed but have not been through the copyediting, formatting, or proofreading process.

1 Hippocampal reactivation extends for several hours following

2 novel experience

5 Bapun Giri (1,2), Hiroyuki Miyawaki (1,3), Kenji Mizuseki (3,4), Sen Cheng (5),

6 Kamran Diba (1,2)

7 (1) Dept of Psychology, University of Wisconsin-Milwaukee, PO Box 413,

8 Milwaukee, WI 53201

9 (2) Dept of Anesthesiology, 1150 W Medical Center Dr, University of Michigan, Ann

10 Arbor, MI 48109

11 (3) Dept of Physiology, Osaka City University Grad School of Medicine, Asahimachi

12 1-4-3, Abeno-ku, Osaka, 545-8585, JAPAN

13 (4) Center for Molecular and Behavioral Neuroscience, Rutgers University, Newark,

14 NJ 07102

15 (5) Institut fur Neuroinformatik, Ruhr-University Bochum, Bochum, Germany

17 Correspondence should be addressed to KD: [email protected]

21 Abstract (190/250 words)

22 New memories are believed to be consolidated over several hours of post-task sleep. The

23 reactivation or “replay” of hippocampal cell assemblies has been proposed to provide a

24 key mechanism for this process. However, previous studies have indicated that such

25 replay is restricted to the first 10-30 minutes of post-task sleep, suggesting it has a limited

26 role in memory consolidation. We performed long-duration recordings in sleeping and

27 behaving male rats and applied methods for evaluating the reactivation of neurons in

28 pairs as well as in larger ensembles while controlling for the continued activation of

29 ensembles already present during pre-task sleep (“preplay”). We found that cell

30 assemblies reactivate for up to 10 hours, with a half-maximum timescale of ~ 6 hours, in

31 sleep following novel experience, even when corrected for preplay. We further confirmed

32 similarly prolonged reactivation in post-task sleep of rats in other datasets that used

33 behavior in novel environments. In contrast, we saw limited reactivation in sleep

34 following behavior in familiar environments. Overall, our findings reconcile the duration

35 of replay with the timescale attributed to cellular memory consolidation and provide

36 strong support for an integral role of replay in memory.

38 Significance Statement (97/120 words)

39 Neurons that are active during an experience, reactivate again afterwards during rest and

40 sleep. This replay of ensembles of neurons has been proposed to help strengthen

41 memories, but it has also been reported that replay occurs only in the first 10-30 minutes

42 of sleep, suggesting a circumscribed role. We performed long-duration recordings in the

43 hippocampus of rats and found that replay persists for several hours in sleep following

44 novel experience—far beyond the limits found in previous reports based on shorter

45 recordings. These findings reconcile the duration of replay with the hours-long timescales

46 attributed to memory consolidation.

48 Introduction (244/650 words)

49 Sleep is necessary for memory consolidation (Rasch and Born, 2013); disruption of

50 sleep for 3-6 h following hippocampus-dependent tasks impairs the formation of long-

51 term memories (Havekes and Abel, 2017). One possible mechanism for memory

52 consolidation in sleep is replay, the process by which neuronal assemblies active during

53 tasks reactivate during subsequent sleep (Marr, 1971; Buzsaki, 2015). However, previous

54 accounts have indicated that replay is limited to the first 10-30 minutes of sleep following

55 a task (Wilson and McNaughton, 1994; Kudrimoti et al., 1999; Tatsuno et al., 2006; Ji

56 and Wilson, 2007). This disparity between the short duration of replay and the long time

57 course of sleep’s effects on memory has presented a major challenge for the field, leading

58 many to argue that replay cannot effectively serve as a mechanism for memory

59 consolidation (Sutherland et al., 2010; Tononi and Cirelli, 2014). However, a careful

60 inspection of previous studies reveals that replay was largely tested in well-trained

61 animals following memory tasks or exploration in familiar environments and that most

62 studies did not maintain unit recordings beyond the first hour of post-task sleep (POST)

63 (Havekes and Abel, 2017). Since novel experience is a powerful trigger for plasticity and

64 learning in the hippocampus (van de Ven et al., 2016; Bittner et al., 2017), we re-

65 examined the extent and duration of reactivation of rat hippocampal CA1 neurons in

66 long-duration recordings following experiments in which both the task and the

67 environment were designed to be novel experiences for the animal.

68 Materials and Methods

69 Surgery

70 Data from a total of ten Long Evans male rats weighing 250-400 g were analyzed in

71 this study, and were previously used in other studies (Mizuseki et al., 2009; Grosmark

72 and Buzsaki, 2016; Miyawaki and Diba, 2016). We will hereby refer to these individually

73 as the Miyawaki, Mizuseki and Grosmark datasets. Protocols originally used to obtain

74 these data were approved by the Animal Care and Use Committees at the University of

75 Wisconsin-Milwaukee, Rutgers University-Newark, and New York University,

76 respectively.

77 All surgeries were performed in a stereotaxic frame under isoflurane anesthesia, with

78 recording started only after full recovery (> 5 days) from surgery. In all three datasets,

79 recordings were performed using implanted 32-channel and 64-channel “Buzsaki” and

80 “Buzsaki-sp” silicon probes (Neuronexus, MI). The Miyawaki dataset was obtained with

81 a Neuralynx data-acquisition system (Neuralynx, MT) at 30 and 32 kHz sampling rate;

82 the Mizuseki recordings used a DataMax system (RC Electronics, MT) at 20kHz

83 sampling rate; Grosmark recordings were performed on Amplipex recording systems

84 (Amplipex ltd., Hungary) sampling at 20 kHz. All datasets were recorded from the CA1

85 layer of the hippocampus with two stainless steel screws implanted above the cerebellum

86 used for referencing and grounding. Mizuseki also recorded simultaneously from the

87 medial entorhinal cortex, which was not used in the present study. In addition, Miyawaki

88 and Grosmark obtained the electromyogram (EMG) using wires placed in nuchal

89 muscles. In all datasets, behavior was monitored and position of head-mounted LEDs was

90 tracked using an overhead digital camera. In all datasets, spikes were detected by filtering

91 and thresholding the wide-band signal using custom software in NDManager and its

92 plugins (Hazan et al., 2006) (http://ndmanager.sourceforge.net) and automatically sorted

93 with KlustaKwik (Harris et al., 2000) (http://klusta-team.github.io/klustakwik), followed

94 by manual inspection and reclustering using the Klusters package (Hazan et al., 2006).

95 All data underwent additional manual inspection and reclustering to ensure consistent

96 standards across datasets.(Csicsvari et al., 1998; Bartho et al., 2004)

97 All three datasets featured recordings sessions with sleep and rest before and after

98 (“PRE” and “POST”) a maze (“MAZE”) epoch. Recordings were paused for between 3-

99 45 min in between these epochs (to untangle recording cables, prepare setups, transfer

100 animals, etc.). Additional criteria were implemented for all recordings to ensure unit

101 stability across PRE and POST periods. Any units whose isolation distance changed by >

102 50% across PRE-POST (Schmitzer-Torbert et al., 2005) were excluded from the

103 analyses. Additionally, any units whose firing rates dropped below 30% of the mean

104 firing rate for combined PRE and POST in any 1 h time window were also excluded.

105 Units were categorized into pyramidal or interneurons based on their wave shape,

106 refractory period, firing rates and burstiness (Csicsvari et al., 1998; Bartho et al., 2004).

107 Behavior

108 Three of the animals (rats R, T, and K) from the Miyawaki dataset, which provided

109 initial data used to generate Figures 1, 2, and 4 (individual sessions shown in Figure 1-1),

110 were trained to drink from water wells while water-restricted (30 mins ad libitum water

111 per 24 h) but had not been placed on any track environments. Following recovery from

112 surgery for electrode implantation, animals were again water-restricted to motivate

113 running on the track. Recording in the home cage commenced ~6 h into the dark cycle

114 (PRE). Three hours before the start of light cycle, animals were transferred to the track

115 (MAZE). Water rewards were supplied at either ends of the track to motivate running.

116 For rats R and K, each extended track running session was composed of two consecutive

117 blocks. In the first block, during the first 20 trials, the track was obstructed to confine the

118 animal to the reward platform for 2 minutes after each trial. After 20 trials, the second

119 block began, during which animals could freely explore the track and platforms for water

120 rewards. For rat R, recordings were performed on this same track for two additional days.

121 These additional track sessions were not considered novel experience and were not

122 pooled with the other data. In the third animal, rat T, no obstruction block was performed

123 and the rat ran freely for the entire session. Three recording sessions were performed with

124 this animal with the shape of the track modified from I to L to U on consecutive days; all

125 of these sessions were considered novel experience. The Miyawaki track sessions lasted

126 ~3 h. Following the track sessions, animals were returned to the home cage and

127 recordings (POST) resumed during their sleep and waking rest. Animals had ad libitum

128 access to food in the home cage. To compare correlations across entire light and dark

129 cycles, we also identified and isolated among these data a total of twenty long-duration

130 sessions from five Long Evans male rats spent entirely in the home cage; these sessions

131 are individually depicted in Figure 7-1.

132 As described by Grosmark and Buzsaki (2016), the four animals from the Grosmark

133 dataset “were pre-trained to search for water on a geometrically unrelated open-field

134 'cheese-board' maze for several day before novelty maze sessions. Once electrodes

135 reached the CA1 pyramidal layer and the animals were well acclimatized to running for

136 water reward as well as to the ‘familiar’ room as determined by the observation that the

137 animals engaged in uninterrupted sleep in this room, a ‘novelty’ session was recorded. A

138 novelty session consisted of a ‘PRE’ epoch in the familiar room, a novelty run in one of

139 the three novel rooms and ‘POST’ epoch back in the familiar room. Only one novelty

140 room was used per novelty session. Note also that […] in the present study the animals

141 had never been inside of the novel rooms, ensuring that the animals had no experience of

142 the maze context, even fleeting ones during the plugging of the electrophysiological

143 headstages, prior to novelty exposure.” These animals received water rewards for

144 completing runs across linear or circular mazes. “The RUN sessions were terminated

145 once the animals were satiated and no longer ran for reward.” (Grosmark and Buzsaki,

146 2016). The recordings were all performed during the light cycle.

147 In the Mizuseki dataset, animals were water-restricted for ~ 24 h prior to experiments

148 and had been previously trained to alternate between two arms of a figure with additional

149 running on a running wheel in a delay area (Pastalkova et al., 2008; Mizuseki et al.,

150 2009). One of these sessions was recorded on the first day of exposure (for 65 min) to the

151 linear track (“novel”), though the animal had previously performed the alternation task

152 and had multiple days of experience in the recording room. The other sessions were

153 performed after at least 10 days of exposure to the linear track (“familiar”) placed at

154 varying orientations in the room. Sleep and rest before (PRE) and after (POST) the maze

155 epochs were recorded. These recordings were all performed during the light cycle.

156 Sleep scoring

157 In the Miyawaki dataset, sleep scoring was performed using EMG, movement and

158 theta-delta power. EMG was smoothed with a 1-s Gaussian filter and z-scored and a

159 Schmitt-trigger threshold set at 0 and 0.5 was used to detect state transitions between low

160 and high EMG power. Periods with low and high EMG power were labelled as sleep and

161 wake respectively. The theta (5-10 Hz) over delta (1-4 Hz) plus (10-14 Hz) band ratio of

162 the power spectral density was used to detect transitions between high theta and low

163 theta, using custom MATLAB software, followed by visual inspection. Sleep states with

164 high theta were classified as REM (rapid eye movement) and the remainder were

165 classified as non-REM. Wake periods with high theta were labeled as “active” and the

166 remaining were labeled “quiet”.

167 In the Grosmark dataset (Grosmark et al., 2016), “sleep scoring was performed using

168 hippocampal LFP (theta/delta ratio), accelerometer (movement), and EMG data” and “all

169 state scoring was performed using TheStateEditor developed by Andres Grosmark in the

170 Buzsaki lab

171 (https://github.com/buzsakilab/buzcoderough/tree/master/BehavioralStateDetection)”.

172 Active wake was “characterized by high theta-delta ratio and active movement/EMG-

173 activity”; quiet wake (labeled “drowsy” by Grosmark) was “characterized by low overall

174 spectral power, low movement/EMG-activity”; NREM was “characterized by high

175 delta/theta ratio, and very low movement/EMG activity”; periods labeled as

176 “intermediate” by Grosmark “ which are short (~30 second) states which occur at the

177 NREM to REM transition and are characterized by highly-elevated pyramidal layer

178 spindle (12 to 20Hz) power and low movement/EMG-activity” were considered as part of

179 NREM here; REM was “characterized by high theta/delta ratio and very low

180 movement/EMG activity occurring after NREM episodes” (Grosmark et al., 2016).

181 In the Mizuseki dataset, sleep versus wake were scored by visual inspection of the

182 animal and the local field potentials from CA1 and the entorhinal cortex while the animal

183 was in its home cage. Quiet and active wake were not further separated. Theta (5-11 Hz)

184 over delta (1-4 Hz) plus (12-14 Hz) band ratio of the power spectral density was used to

185 detect transitions between high and low theta using custom MATLAB software. Sleep

186 states with high theta were classified as REM (rapid eye movement) and the remainder

187 were classified as non-REM. These periods were further “cross-validated with

188 experimenter notes taken while observing theta activity on-line in sleep session and

189 verifying that the rat was sleeping” (Mizuseki et al., 2011).

190 Experimental Design and Statistical Analysis

191 A total of sixteen sessions composed of PRE/MAZE/POST recordings from ten male

192 Long Evans rats were used. Statistical comparisons between POST and PRE were

193 performed in each animal separately, as described in detail below, with the sample size

194 depending on the number of isolated putative pyramidal neurons for each session. All

195 individual sessions are shown in Figure 1-1 (including numbers of putative pyramidal

196 neurons recorded in each session) to demonstrate consistency of findings across animals

197 and datasets. Pooled results are shown in Fig. 1b, Fig. 2c, and Fig. 3b, with error bars

198 indicating standard error of the mean pooled over sessions. Statistical analyses were

199 performed using MATLAB. All correlations shown in figures were calculated using

200 Pearson’s r and significance level was set at D = 0.001. Exact p-values are provided in

201 the Results section.

202 Explained Variance Measure

203 Spike times were binned into 250 ms time bins, creating an N x T matrix, where N is

204 the number of neurons and T is the number of time bins. The time bin size was chosen

205 based on a previous study (Tatsuno et al., 2006) and because it matches the timescales for

206 ~ 2 theta cycle oscillations and sharp-wave ripples. Pearson’s correlations, R, were

207 determined for spike counts from neuronal pairs recorded on different shanks in 15 mins

208 sliding windows (window length 15 min, sliding 5 min steps) in PRE and POST and in

209 the entire MAZE session to produce P, an M-dimensional vector, where M is the number

210 of cell pairs. To assess similarity between P vectors from different windows, the Pearson

211 correlation R of these vectors (i.e. the correlation between cell pair correlations) was

212 determined (e.g. R[PRE,POST], R [PRE,MAZE] and R[MAZE,POST]). The main element of interest

213 here is R[MAZE,POST], but it may be contaminated by preexisting correlations before the

214 animal has been exposed to a maze (Kudrimoti et al., 1999; Ribeiro et al., 2004; Tatsuno

215 et al., 2006). To address this issue, we calculated the explained variance for a window

216 WIN, based on the square of partial correlation (Kudrimoti et al., 1999):

ଶ ሾ୑୅୞୉ǡ୛୍୒ሿ െሾǡሺሻሿሾሺሻǡ ሿ ܧܸሺ ሻ ൌ ൭ ൱ ଶ ଶ ξሺͳ െ ሾǡሺሻሿሻξሺͳ െ ሾሺሻǡ ሿሻ

217 averaged over all PRE(k) windows for which WIN z PRE(k). To further assess how

218 much of EV can be generated by chance, we also calculated the time-reversed explained

219 variance (REV) for each window WIN, as proposed in a previous study (Tatsuno et al.,

220 2006):

ଶ ሾǡሺሻሿ െሾ୑୅୞୉ǡ୛୍୒ሿሾሺሻǡ ሿ ܴܧܸሺ ሻ ൌ ൭ ൱ ଶ ଶ ξሺͳ െ ሾǡሺሻሿሻξሺͳ െ ሾሺሻǡ ሿሻ

221 averaged over all PRE(k) windows for which WIN z PRE(k). Error bars for EV and REV

222 in Fig. 1a, Fig. 3a, Fig. 4a, Fig. 5, Fig. 6, Fig. 7a and Figure 1-1 indicate standard

223 deviation over PRE(k). For results pooled across animals and sessions, as shown in Fig.

224 1b and Fig. 3b, error bars indicate standard error of the mean pooled over sessions. For

225 Fig. 1c-e analyses involving pooled pairwise relationships, NREM periods within PRE

226 and POST were concatenated in each session and pairwise correlation vectors were

227 generated. These vectors were pooled across sessions for PRENREM, MAZE and

228 POSTNREM. Residuals for POSTNREM and MAZE (Fig. 1e) were obtained from the

229 respective regression lines with PRENREM.

230 For Fig. 4 analyses on the amount of exposure required to produce significant

231 reactivation, P of MAZE was calculated from spike taken from the first 1 min, 5 min, 15

232 min, 30 min or entire period of exploration in the MAZE session. To extract a timescale

233 for reactivation from sessions which showed some evidence of decay in EV, we first

234 measured the maximum EV across all 15 min (sliding) windows. We then determined the

235 best-fit line through each window which was composed of at least 50% NREM sleep,

236 starting up to 1 h before the maximum EV until the end of the session. The time point at

237 which the EV dropped to half of the maximum EV was then obtained from the regression

238 line (as shown in Figure 1-1).

239 Temporal Bias Measure

240 The temporal bias was calculated from the cross correlogram for each cell pair

241 (calculated with 1 ms bins) by taking the sum of the counts from +1 to +125 ms minus

242 the counts from -1 to -125 ms. (Skaggs and McNaughton, 1996). To compare the

243 temporal order across all pairs in POST and PRE versus MAZE, we took the difference

244 between the number of pairs in the upper-right and lower-left quadrants of the scatter

245 plots (i.e. “positively related” in Fig. 1f scatter plots) and the number of pairs in the

246 upper-left and lower-right quadrants (i.e. “negatively related”). This population temporal

247 bias was greater in POST versus PRE. To determine the significance of this difference

248 between POST and PRE (Fig. 1g), we compared the observed difference value against a

249 null distribution obtained from 10000 independent shuffles of the cell pair identities in

250 PRE and POST.

251 Cell Assembly Reactivation

252 To investigate cell assembly reactivation, we employed a recently developed

253 technique (Lopes-dos-Santos et al., 2013; Trouche et al., 2016) based on independent

254 component analysis (ICA) of spike trains projected onto bases obtained from principal

255 component analysis (PCA). Briefly, a correlation matrix was generated using the N x T

256 matrix of neuronal spike counts in 250 ms windows after it was z-scored. Principal

257 components were calculated following eigenvalue decomposition. To eliminate spurious

258 activity patterns, only principal components having eigenvalues above the Marcenko–

259 Pastur distribution were retained. The z-scored activity matrix was then projected on the

260 subspace spanned by these principal components. The resulting reduced activity matrix

261 was then used to generate the ICA weight matrix. Each neuron received a weight for the

262 corresponding independent component (assembly pattern). Given the random values of

263 weights and their sign, each component was normalized to have sum 1. The dot product

264 of each z-scored bin with the weight vector was then used to calculate the reactivation

265 strength of the assembly pattern. An average reactivation strength was obtained by

266 averaging over all of the independent components in a given time window.

267 Results

268 To examine reactivation following novel experience, we pre-trained rats to alternate

269 drinking from two water wells in their home cages. On the first day of recording, in

270 generating the “Miyawaki dataset”, the rats were placed on a linear track with water wells

271 on platforms at each end of the track. These animals had not previously experienced any

272 linear tracks or other large mazes, which ensured that running for water across a track

273 was a very novel experience on the first day. On subsequent days, the length and shape of

274 the track were modified with new segments, but the task itself was not novel. The track

275 sessions on the first day and on subsequent days showed similar results and were

276 combined in these analyses. For all analyses, we included only putative pyramidal units

277 that displayed stable and isolated clusters (see Materials and Methods). Each recording

278 session was divided into a MAZE period (for the task itself) as well as PRE and POST

279 periods (for sleep or rest in the home cage before and after the task).

280 As co-firing is considered to bind neurons into ensembles and drives synaptic

281 plasticity, we first examined co-activity in pairs of neurons (Fig. 1a-b); we employed a

282 commonly-used “explained variance” (EV) measure (Kudrimoti et al., 1999; Tatsuno et

283 al., 2006; Johnson et al., 2010) to test whether co-activity in 250 ms time windows during

284 MAZE persists in POST beyond the levels present in PRE. This control was necessary

285 because cell pairs showed significant correlations (Pearson’s r = 0.176, p = 5.386 × 10-28,

286 num. of pairs = 3809) between PRE and MAZE (Tatsuno et al., 2006; Dragoi and

287 Tonegawa, 2011; 2013; Fig. 1c) and fairly strong correlations (Pearson’s r = 0.508, p =

288 7.482 × 10-249, num. of pairs = 3809) between PRE and POST sleep (Tatsuno et al., 2006;

289 Dragoi and Tonegawa, 2014; Fig. 1d). However, after controlling for existing PRE

290 correlations, regression analysis revealed a significant correlation (Pearson’s r = 0.448, p

291 = 3.219 × 10-187, num. of pairs = 3809) between MAZE activation and POST non-rapid

292 eye movement (NREM) sleep patterns (Fig. 1e). The time window used in this measure

293 was chosen to correspond to the duration of typical hippocampal sharp-wave ripples and

294 approximately 2 theta oscillation cycles; therefore, EV captures neuronal activation and

295 reactivation compressed within theta sequences and sharp-wave ripple sequences. The

296 mean reactivation was higher in NREM compared to quiet waking rest (EV(NREM) =

297 0.201 ± 0.1464, EV(quiet wake) = 0.083 ± 0.06, p = 0.043). In all sessions involving

298 exploration of a novel track segment we observed elevated reactivation for the entire ~3 h

299 duration of POST (see individual sessions in Figure 1-1), with surprisingly little evidence

300 of decay over this time (Fig. 1b).

301 We and others previously reported that firing rates of hippocampal neurons increase

302 after novel waking experience (Karlsson and Frank, 2008; Larkin et al., 2014; Miyawaki

303 and Diba, 2016), with higher firing rates in POST compared to PRE (Miyawaki and Diba,

304 2016). To rule out that these firing-rate differences account for the increased EV in POST

305 compared to PRE, we repeated this analysis after sub-sampling to equalize the firing rate

306 for each neuron in PRE and POST but found nearly identical results. In particular, the

307 correlation between MAZE and POST NREM, partialed out for PRE NREM, remained

308 significant (median Pearson’s r = 0.454, median p = 6.417 × 10-197 for 100 different

309 random subsamples equalizing PRE and POST spike counts of each cell; num. of pairs =

310 3809). We also redid the analyses with different size time bins, ranging from 50 ms to

311 500 ms, for calculating correlation. These also produced highly similar and consistent

-248 -245 312 results (Pearson’s r50ms = 0.507, p = 5.916 × 10 , r100ms = 0.505, p = 3.968 × 10 ,

-119 313 r500ms = 0.3633, p = 3.420 × 10 ; num. of pairs = 3809), confirming the robustness of

314 the observation.

315 To examine whether the reactivation in POST is consistent with temporal sequences

316 in the firing patterns of neurons (Skaggs and McNaughton, 1996; Lee and Wilson, 2002;

317 Ji and Wilson, 2007), we employed methods introduced by Skaggs and McNaughton

318 (1996) to evaluate temporally ordered replay. We measured the temporal bias in the spike

319 times of neuronal pairs (the difference in the number of positive lag (+1 to +125 ms) and

320 negative lag (-1 to -125 ms) spike counts in the cross correlogram) in PRE, POST and

321 MAZE epochs (see Materials and Methods). The temporal bias in POST was

322 significantly correlated to the temporal bias in MAZE (Fig. 1f; Pearson’s r =0.274, p =

323 1.1 x × 10-66, n = 3809 cell pairs). The temporal bias correlation between PRE and MAZE

324 was weaker, yet also significant (Pearson’s r = 0.053, p = 0.0011), consistent with

325 previous reports (Dragoi and Tonegawa, 2011, 2013). Importantly, in POST compared to

326 PRE, there was a greater concentration of cell pairs that showed a bias consistent with the

327 MAZE patterns (number of points in upper-right and lower-left quadrants minus points in

328 the upper-left and lower-right quadrants of the scatter plots in Fig. 1f; p = 0.0010 for

329 POST — PRE compared to 10000 shuffles), indicating that the temporal spiking patterns

330 in POST more strongly resemble the patterns observed during MAZE (Skaggs and

331 McNaughton, 1996; Wikenheiser and Redish, 2013). Our results therefore demonstrate

332 that the neuronal firing patterns in post-task sleep are consistent with a temporal replay of

333 the sequences experienced during behavior.

334 To extend our analyses to ensembles of neurons and track their reactivation at a finer

335 temporal resolution, we employed a recent technique that identifies co-activate cell

336 assemblies using independent component analysis (ICA) of neural population vectors

337 (Lopes-dos-Santos et al., 2013; Trouche et al., 2016). The resulting ICA weight vectors

338 were extracted from the MAZE period and their reactivation (or pre-activation) was

339 examined in both POST and PRE (Fig. 2a). In both the sample session and pooled

340 sessions, cell assemblies showed greater activation in POST compared to PRE (Fig.

341 2b,c). As expected, these reactivations occurred in conjunction with sharp-wave ripple

342 oscillations in the CA1 pyramidal layer (Buzsaki, 2015) (Fig. 2d). Importantly, the

343 average of the POST sleep cell assembly reactivations lasted well beyond previously

344 reported limits and well into the third hour of post-task sleep, in agreement with results

345 from the EV analysis.

346 Because these sessions involved long ~3 h durations of waking experience, we asked

347 whether briefer exploration in a novel maze would lead to similarly prolonged replay. To

348 answer this question, we extended these same analyses to the Grosmark dataset,

349 generously provided by Grosmark et. al. (2016), with sessions composed of PRE and

350 POST sleep before and after ~45 mins (range: 34-51 mins) of MAZE experience where

351 both the track and the recording room were designed to be novel. Fig. 3a demonstrates a

352 sample recording where POST replay is seen for > 4 h following MAZE. In the pooled

353 Grosmark et. al. (2016) sessions as well, replay was prolonged for several hours in POST

354 sleep in both EV and ICA measures (Fig. 3b). Based on the half-maxima of the EV in

355 these individual sessions, reactivation persisted for a mean of 2.3 h (range = 1.9 – 2.85 h)

356 following MAZE. Thus, ~45 minutes of experience on a novel track was sufficient to

357 promote long-lasting hippocampal replay, well beyond previously-reported limits.

358 During early exploration of a novel environment, neuronal ensembles composed of

359 place-fields are still in the process of forming and stabilizing (Frank et al., 2004; Cheng

360 and Frank, 2008; Feng et al., 2015). We therefore asked whether even early MAZE

361 experience could leave a lasting trace in POST sleep. To examine this question, we used

362 only the first 1-min, 5-min, 15-min or 30-min initial segments of exploration to define the

363 pairwise correlation structure of MAZE and examined whether we could detect

364 significant reactivation of those early ensembles in POST. While POST reactivation more

365 strongly resembles MAZE ensembles defined over longer periods (Fig. 4 a-b), we found

366 that the activation of cell pairs in just the first 5-min of MAZE was sufficient to produce

367 significant reactivation in POST. This observation therefore indicates that ensembles that

368 already emerge within the first few minutes of novel experience continue to reactivate

369 during post-task sleep.

370 In previous reports, reactivation following experience in familiar environments was

371 found only in the first 10-30 mins of sleep (Wilson and McNaughton, 1994; Kudrimoti et

372 al., 1999; Tatsuno et al., 2006; Ji and Wilson, 2007). To examine the role of novelty vs.

373 familiarity, in one animal we recorded PRE, MAZE, and POST over three consecutive

374 days of experience on the same linear track (Fig 5). Replay was lower but still enhanced

375 during POST after the second day of track running (Fig 5, middle panel). However, by

376 the third day on the track, the observed replay was essentially at the chance level (Fig. 5,

377 right panel). To examine this question in additional animals, we analyzed another 4

378 sessions in 3 rats from the Mizuseki dataset (Mizuseki et al., 2009). In one of these

379 sessions, the animal was newly introduced to the linear track, but had previously

380 performed delayed alternation in a figure eight maze with a running wheel (Pastalkova et

381 al., 2008; Mizuseki et al., 2009) in this recording room. Remarkably consistent with the

382 Miyawaki sessions, replay persisted for over 3 h in POST sleep following this novel

383 experience (Fig. 6a; time of half-maximum = 2.22 h). In contrast, in 3 other sessions

384 recorded from 2 animals after extensive exposure to the track (Fig. 6b), replay was much

385 more limited and was significant for < 30 min (range of 15-30 min, mean =23.33 min,

386 Student’s t test one tailed paired p<0.001) of POST NREM sleep. Similar results were

387 seen when we assessed reactivation in the ICA-defined cell assemblies (Fig. 6c-d). In

388 sum our observations indicate that the novelty of the environment experienced during

389 spatial behavior is a key determinant of the duration of reactivation in the subsequent

390 sleep.

391 To better determine the temporal extent of reactivation following novel experience in

392 MAZE, in two sessions from the Miyawaki dataset we were able to extend our recordings

393 for ten hours of POST sleep (Fig. 7a). In both these sessions, reactivation peaked at ~3-5

394 h following MAZE, then showed progressive decrease, dropping past half-maximum at ~

395 6.28 h into POST sleep. These sessions appeared to feature longer reactivation than the

396 Grosmark sessions. Thus, it may be that the longer MAZE periods in our sessions led to a

397 longer reactivation period. However, the longer replay we observed could also be

398 attributed to the more continuous sleep we obtained in our light-cycle recordings

399 compared to Grosmark et. al. (2016) (e.g. compare fractions of sleep between Fig. 1b and

400 Fig. 3b) while fragmentation of sleep may limit the persistence of reactivation.

401 Nevertheless, in all three datasets of recordings examined, we found that reactivation

402 extended for several hours following novel experience.

403 We next examined the activation patterns of neurons over the course of sleep in the

404 correlations between neuronal pairs across in our long-duration recordings (15-min bins).

405 Several interesting features can be gleaned from these figures (Fig. 7b). As noted

406 previously, there are large correlations between all recorded sleep periods, including

407 those that extend from PRE to POST sleep. The MAZE patterns are largely divergent

408 from those during sleep, though some correlations were still noted between PRE and

409 MAZE (e.g. Fig. 1c). Thus, the MAZE experience lead to a correlation pattern that is

410 largely unique. While these patterns persist in POST sleep, compared to PRE, they occur

411 alongside the stronger patterns that are in turn unique to sleep. As can also be seen in Fig.

412 7b, the correlation structure during POST shifts over time so that different neuronal

413 groups are dominant at different times. For example, relatively weaker apparent

414 correlations exist across than within the end and beginning of the Day 3 session recorded

415 from Rat T, with the dominant patterns shifting over hours-long periods. Additional long-

416 duration sessions (n=20 long 6-12 h light or dark cycle sessions across 5 animals; see

417 Figure 7-1) further corroborate this viewpoint.

418 Discussion

419 These results demonstrate that cell assemblies associated during learning reactivate

420 during subsequent sharp-wave ripples for a much longer timespan than has been

421 generally appreciated. In particular, we found that cell assemblies in sleep reprise MAZE

422 activity for up to 7 hours following novel experience. Remarkably, similar timescales

423 have been reported both for the post-task consolidation window during which sleep

424 deprivation impairs memory formation in different hippocampus-dependent learning

425 tasks (Havekes and Abel, 2017). For example, deprivation of sleep in the 4-5 hours after

426 object-location exploration disrupted subsequent memory (Florian et al., 2011; Havekes

427 et al., 2014; Prince et al., 2014). Importantly, this memory deficit was observed even

428 when the first hour of post-task sleep, and presumably concurrent replay, was left intact

429 (Prince et al., 2014), indicating that processes beyond the first hour of sleep remain

430 important for memory consolidation. Other studies investigating contextual fear

431 conditioning (Graves et al., 2003; Vecsey et al., 2009) and object-recognition memory

432 (Palchykova et al., 2006) reported a similar 5-hour post-task consolidation window,

433 consistent with our report. Our results are also consistent with a recent report that

434 enduring stability of neural ensemble patterns supported by parvalbumin-positive

435 interneurons (Ognjanovski et al., 2017) following contextual fear conditioning.

436 The duration of replay we report here also closely matches the hours-long timescales

437 reported for protein signaling following memory (often referred to as “cellular” or

438 “synaptic” consolidation (McGaugh, 2000; Dudai, 2004)). In particular, cyclic adenosine

439 monophosphate (cAMP) and protein kinase A (PKA) signaling pathways, which regulate

440 the cAMP response element binding protein (CREB), are critically involved in multiple

441 forms of synaptic plasticity linked to memory, as well as in gene expression in the hours

442 immediately following learning (Abel et al., 2013). Importantly, these pathways are

443 impaired by 5 hours of sleep-deprivation (Bourtchouladze et al., 1998; Vecsey et al.,

444 2009). Protein synthesis through the mammalian target of rapamycin (mTOR) kinase

445 complex, which is likewise implicated in both plasticity and memory formation (Hoeffer

446 and Klann, 2010), is also reduced upon sleep-deprivation over a 5-h post-task window

447 (Vecsey et al., 2012; Tudor et al., 2016). While more research is needed to understand the

448 inter-relationship between reactivation and protein-signaling, these results help to

449 reconcile previous discrepancies between the time windows considered for

450 cellular/synaptic and systems consolidation of memory.

451 We used two different methods for evaluating and tracking reactivation in POST

452 relative to PRE: the EV method based on pairwise correlations (Kudrimoti et al., 1999;

453 Tatsuno et al., 2006) and an ICA method based on ensemble co-activity (Peyrache et al.,

454 2009; Lopes-dos-Santos et al., 2011; Lopes-dos-Santos et al., 2013). While these methods

455 are not designed to evaluate temporal sequences, we used a 250-ms time window to

456 capture co-activation on the timescale of sharp-wave ripples during which replay

457 sequences are observed (Nadasdy et al., 1999; Buzsaki, 2015). Furthermore, we

458 examined the temporal structure of spike times within this window using a third method,

459 by comparing the temporal bias of spike times in pairs of neurons, and found that POST

460 firing sequences showed significant fidelity to MAZE sequences and greater temporal

461 compared with PRE sequences, consistent with previous accounts of temporal replay

462 (Skaggs and McNaughton, 1996; Lee and Wilson, 2002; Wikenheiser and Redish, 2013).

463 Preliminary analysis of our data with additional methods that are sensitive to sequential

464 structure at the population level (Maboudi et al., 2018b) have also produced consistent

465 results to what we report here (Maboudi et al., 2018a) (see also Chen et al., 2016), but

466 further work is needed to evaluate and differentiate temporal relationships across PRE,

467 MAZE, and POST.

468 Importantly, previously studies by different investigators, some using methods

469 identical to ours, had previously observed much shorter durations for replay than we

470 report here (Wilson and McNaughton, 1994; Kudrimoti et al., 1999; Tatsuno et al., 2006;

471 Ji and Wilson, 2007). This discrepancy may arise from two reasons. First, with the

472 exception of Tatsuno et. al. (2006), these studies did not examine reactivation beyond the

473 first 1 h of post-task sleep as we have, potentially missing re-emergent reactivation in

474 subsequent sleep. Second, the MAZE experience in these studies involved exploration of

475 environments that were highly familiar to the animal, whereas we observed that the time-

476 course of reactivation depended on the novelty of the task environment. Interestingly,

477 Tatsuno et. al. (2006) introduced novel objects to animals in a familiar environment but

478 failed to find evidence of protracted replay. This was in contrast to Ribeiro et. al. (2004)

479 who performed similar experiments that appeared to show reactivation lasting for days

480 but did not control for extant correlation in PRE. Our analysis (in Fig. 1c,d) demonstrates

481 the importance of these controls, in agreement with Tatsuno et. al. (2006), which may

482 otherwise give an illusion of replay. Overall, our findings indicate that prolonged

483 reactivation in post-task sleep is contingent upon exposure to a novel environment, which

484 induces a new hippocampal map (or “global remapping”), rather than the experience of

485 novel objects or events in a familiar environment, for which the hippocampus makes use

486 of the same spatial map (or “rate remapping” (Muller and Kubie, 1987; Leutgeb et al.,

487 2005)). An intriguing possibility is that other experiences, such as a shock (Moita et al.,

488 2004) or exposure to a predator’s odor (Wang et al., 2012), which produce “global

489 remapping” can generate a similarly prolonged reactivation in subsequent sleep.

490 The long duration of replay demonstrated in our study counters the notion that the

491 bulk of neuronal patterns in sleep following learning are random or noisy, and

492 strengthens the argument that memory can be consolidated during sleep through a

493 systems level process involving hippocampal replay (Buzsaki, 1989; Diekelmann and

494 Born, 2010). We propose that novelty increases the firing rates of those hippocampal

495 neurons activated in the environment (Hirase et al., 2001) through increased membrane

496 excitability alongside potentiation of synapses that bind the activated cell assemblies

497 (Takeuchi et al., 2014). Some of these processes are likely enhanced by activation of

498 membrane dopaminergic receptors (Takeuchi et al., 2016), which promotes the

499 reactivation of these assemblies in post-task sleep. Over the course of sleep, reactivation

500 during sharp-wave ripples promotes the consolidation of memories (Maingret et al.,

501 2016). But as memories are consolidated, hippocampal firing rates then decrease

502 (Miyawaki and Diba, 2016), likely as a consequence of synaptic downscaling also

503 triggered by sharp-wave ripples (Tononi and Cirelli, 2014; Miyawaki and Diba, 2016;

504 Norimoto et al., 2018). It is interesting and worthwhile to note that while extended replay

505 is clearly a significant phenomenon, even in POST sleep the MAZE patterns are not the

506 most dominant activation patterns. Additionally, a small but significant correlation exists

507 between PRE and MAZE periods, consistent with the notion of “preplay” (Dragoi and

508 Tonegawa, 2011, 2013). Thus, replay must contend with activity patterns that are unique

509 to sleep, including those that carry onto subsequent experience (Dragoi and Tonegawa,

510 2014; Grosmark and Buzsaki, 2016). However, the function carried out by these other

511 (non-replay) activities within sleep still remains elusive. As network, cellular, and

512 synaptic processes are intimately intertwined, additional research is needed to better

513 understand whether and how reactivation and non-reactivation activities during sleep

514 contribute to consolidation and other sleep functions by engaging proteins and plasticity

515 processes to strengthen and weaken synaptic connections.

516 Acknowledgements

517 We are grateful to Asohan Amarasingham, Laura Ball, Caleb Kemere, Andrew P.

518 Maurer, and Brendon Watson for helpful comments, and to Andres Grosmark, John Long

519 and Gyorgy Buzsaki for publicly sharing their valuable data. This project was funded by

520 the National Institute of Mental Health (NIMH) R01MH109170 and R01MH117964 to

521 K.D., with additional support to S.C. from the German BMBF 01GQ1506 and to K.M.

522 from the Toray Science Foundation.

523

524 Data Sharing

525 Data and analyses code will be made available to readers upon reasonable request.

526 Figures and Legends

527 Figure 1. Neuronal reactivation persists for hours following novel experience.

528 a) Reactivation as assessed by explained variance (Kudrimoti et al., 1999) (EV; black, mean ±

529 SD in 15 min bins, sliding in 5-min steps) was significantly enhanced in POST sleep following

530 novel MAZE in a sample recording (37 cells, 497 pairs), relative to the chance level (green;

531 mean + SD) assessed by reversed EV (REV ;Tatsuno et al., 2006; see Materials and Methods)

532 (REV; see Materials and Methods). Detected brain states are indicated in the background. b)

533 Results pooled across 5 POST sessions from 3 animals (Miyawaki dataset; mean ± SEM for n =

534 5 sessions, 205 cells and 3809 pairs; see Figure 1-1 for individual sessions). The total ratio of

535 time in each sleep/wake state is shown in the background for each bin as a proportion of the y-

536 axis limit. c) Neuronal pairwise correlations persist from PRE to MAZE. d) Much stronger

537 correlations are observed across PRE and POST NREM sleep. These relationships are controlled

538 for in the EV measure (Kudrimoti et al., 1999; Tatsuno et al., 2006). e) After regressing out for

539 the correlations with PRE, strong partial correlations are evident between POST NREM and

540 MAZE periods. f) Scatter plots of the temporal bias (total counts in the cross-correlogram from

541 +1 ms to +125 ms minus counts from -1 ms to -125 ms (Skaggs and McNaughton, 1996)) of cell

542 pairs (3809 pairs from 5 sessions) on MAZE versus PRE and POST (same axes scales for both

543 plots). g) The population temporal bias (number of scatter plot points in upper-right and lower-

544 left quadrants minus those in upper-left and lower-right quadrants) was significantly greater in

545 POST compared to PRE (p= 0.001, permutation test of observed value of POST – PRE

546 compared against 10000 shuffles of cell pair identity). ** indicates p < .01 and *** indicates p <

547 .001.

548

549 Figure 2. Assembly reactivation persists for hours following novel experience.

550 a) Replay was assessed using independent component analysis (ICA) of cell assemblies (Lopes-

551 dos-Santos et al., 2013). In a sample recording showing rasters for 37 neurons (right axis)

552 spanning PRE, MAZE, and POST periods, ICA was used to define cell assemblies from the

553 MAZE period (3 out of 10 identified assemblies shown on far right). (Re)Activation strength of

554 these three assemblies (left panel) is shown in corresponding colors. Ripple band local-field

555 potential is shown below each panel. b) (Re)Activation of the third assembly from panel a is

556 shown in PRE (left) and POST (right) panels for the same recording as Fig. 1a, with sleep/wake

557 states shown in the background. The average (re)activation strength of all 10 cell assemblies for

558 this recording is overlaid in red (mean in 15 min windows sliding in 5-min steps; right axis). c)

559 Mean (± SEM) cell assembly reactivation strength pooled across 5 sessions from 3 animals

560 (num. of assemblies = 54; Miyawaki dataset), with the total ratio of time in each sleep/wake state

561 in the background for each bin. Assembly reactivation strength remained significantly above

562 zero throughout 3 hours of POST. d) Assembly reactivation was coincident with hippocampal

563 sharp-wave ripple events.

564

565 Figure 3. Replay following brief novel experience

566 a) Reactivation (EV; mean ± SD from 108 cells, 4984 pairs) in POST following brief (45 mins)

567 novel track experience in a sample session from the Grosmark dataset (Grosmark et al., 2016).

568 Sleep/wake states are indicated in the background. b) Reactivation analysis results pooled across

569 5 POST sessions from 4 animals (total 278 cells, 8810 pairs; see Figure 1-1 for individual

570 Grosmark sessions) shows extended replay for ~3 h following novel experience (mean ± SEM

571 for EV, black, left axis and Assembly reactivation strength, red, right axis). Note that these

572 animals demonstrated less NREM sleep after this point, making it difficult to assess the

573 continuation of replay. The total ratio of time in each sleep/wake state is shown in the

574 background for each bin as a proportion of the y-axis limit. Since individual sessions varied in

575 duration, N varies from 4 to 5 in different bins.

576

577 Figure 4. Reactivation of the onset of MAZE experience.

578 a) We iteratively used the first 1 min, 5 min, 15 min, 30 min or entirety of the behavior session to

579 define the MAZE pairwise correlation template and calculated reactivation (EV; mean ± SD) in

580 POST accordingly (sleep/wake states indicated in background). The strength of reactivation

581 increased with the length of the initial period used to define the template in this sample session,

582 but some reactivation could be seen of even the first 1-min of MAZE. b) Pooled results from EV

583 in NREM in POST (n = 5 sessions from the Miyawaki dataset, with 205 cells and 3809 pairs) for

584 these incremental durations of MAZE experiences. All durations produced significant Pearson

585 partial correlation coefficients (1-min p = 1.82 × 10-13 ; 5-min p =4.835 × 10-32 ; 15-min p =

586 3.647 × 10-57; 30-min p = 3.917 × 10-84; All MAZE p = 3.219 × 10-187).

587

588 Figure 5. Reactivation decreases following repeated exposure to an environment

589 Reactivation (EV; mean ± SD) in POST following repeated exposure of an animal to same

590 environment (rat R, Env 1), on Day 1 (58 cells, 1332; left panel), Day 2 (86 cells, 2966 pairs;

591 middle panel) and Day 3 (83 cells, 2717 pairs; right panel). The strength of reactivation shows a

592 progressive decrease over days of exposure to the environment. Sleep/wake states are indicated

593 in the background.

594

595 Figure 6. Limited reactivation in sleep following behavior in a familiar environment

596 a) Extended reactivation (EV; mean ± SD from 42 cells, 567 pairs) is observed in POST

597 following a 65-min session on a novel track from the Mizuseki dataset. Sleep/wake states are

598 indicated in the background. b) In contrast, very limited reactivation (EV; mean ± SD from 3

599 sessions from the Mizuseki dataset; 44 cells, 248 pairs) is observed in POST following behavior

600 on a highly familiar track (> 10 exposures). Horizontal gray lines along the x-axis indicate

601 intervals during which recording was paused by the experimenter. c) (Re)Activation of a sample

602 ICA assembly (gray) is shown in PRE (left) and POST (right) panels for the Roy Day 3 Env 1

603 session, with sleep/wake states shown in the background. The average (re)activation strength of

604 all 20 cell assemblies for this recording session is overlaid in red (mean in 15 min windows

605 sliding in 5-min steps; right axis). d) Reactivation strength pooled across 4 POST sessions

606 following MAZE in a familiar environment show evidence for little or no reactivation (p = 0.6

607 for all POST; p = 0.74 for NREM only; one-tailed one-sample t-test against mean 0). Error bars

608 denote + SEM.

609

610 Figure 7. Reactivation in long-duration sleep recordings. In long-duration recordings from

611 two sessions from two different animals, a) replay (EV, black, mean ± SD versus REV, green,

612 mean ± SD) persisted for the duration of the recording (timestamp corresponding to half-

613 maximum = 6.55 h for rat R, 58 cells, 1332 pairs and = 6.01 h for rat T, 17 cells, 102 pairs). b)

614 The vector of pairwise correlations was compared across the recordings (from PRE to MAZE to

615 POST in 15-min bins). As noted, strong correlations exist within and across PRE and POST

616 sleep, though correlations with MAZE are stronger in POST for several hours. As sleep

617 progresses, different pairs dominate, with weaker correlations across, rather than within, late

618 sleep and early sleep. See Figure 7-1 for additional sleep sessions across entire light/dark cycles.

619

620

621

622 Figure 1-1. POST reactivation for each session shown separately.

623 a) Explained variance (EV, black; mean ± SD) and time reversed EV (REV, green; mean ± SD)

624 measured in 7 sessions from 3 animals in the Miyawaki dataset. Animal name, recording day,

625 environment label (e.g, Env1) and number of putative pyramidal cells (e.g, 58 pyr) used in the

626 analysis are indicated on top of each panel. Detected brain states are shown in background. Note

627 that all sessions were performed in novel environments (numbered Env1 to Env3), except for

628 RoyDay2Env1 and RoyDay3Env1, which featured repeated exposures to the same maze as

629 RoyDay1Env1 and were therefore not pooled with the others for the novel MAZE analyses in

630 Fig. 1, 2 and 4. b) Similar to (a) shown for the Grosmark dataset (Grosmark et al., 2016).

631 Blackened background in BuddyDay1Env1 indicates a period that contained excessive high-

632 frequency noise that temporarily prevented reliable unit identification. c) Similar to (a) shown for

633 sessions from the Mizuseki dataset. Horizontal gray lines along the x-axis indicate intervals

634 during which the recording was interrupted by the experimenter. In sessions in which

635 reactivation appeared to decay over time, we determined the maximum EV (red circle on y axis),

636 then found the best-fit line (shown in orange) for the EV for windows with > 50% NREM sleep

637 from 1 h before the max EV time point to the end of the session (see Materials and Methods).

638 We then used the best-fit line to determine the time point (time constant) corresponding to the

639 half maximum of the EV.

640

641

642 Figure 7-1. Co-activation persists for several hours during light and dark cycles.

643 Twenty sessions from five different animals freely sleeping/waking in the home cage during

644 light/dark cycles. Animal names, cycle, day of recording and # of putative neurons are provided

645 on top of each correlation matrix. Some of these sessions overlapped with the Miyawaki sessions

646 used in the PRE/POST analyses but were reclustered to track units over the periods shown.

647 Methods were otherwise identical to those used in the Miyawaki dataset as described. Matrices

648 show pairwise correlations of each time window compared across the entire sleep session (15

649 min bins). In each 15-min bin, a vector of all neuronal pairwise correlations was computed; the

650 correlation coefficients of these vectors were then calculated to determine similarity across time.

651 While there was variability across sessions, many sessions feature similar pairwise correlations

652 that persist for several hours that are then replaced by other dominant pairs.

653

654

655 656 657 658

659 References

660 Abel T, Havekes R, Saletin JM, Walker MP (2013) Sleep, plasticity and memory from molecules

661 to whole-brain networks. Curr Biol 23:R774-788. PMCID: PMC4263505 PMID:24028961.

662 Bartho P, Hirase H, Monconduit L, Zugaro M, Harris KD, Buzsaki G (2004) Characterization of

663 neocortical principal cells and interneurons by network interactions and extracellular

664 features. J Neurophysiol 92:600-608 PMID:15056678.

665 Bittner KC, Milstein AD, Grienberger C, Romani S, Magee JC (2017) Behavioral time scale

666 synaptic plasticity underlies CA1 place fields. Science 357:1033-1036 PMID:28883072.

667 Bourtchouladze R, Abel T, Berman N, Gordon R, Lapidus K, Kandel ER (1998) Different

668 training procedures recruit either one or two critical periods for contextual memory

669 consolidation, each of which requires protein synthesis and PKA. Learn Mem 5:365-374.

670 PMCID: PMC311273 PMID:10454361.

671 Buzsaki G (1989) Two-stage model of memory trace formation: a role for "noisy" brain states.

672 Neuroscience 31:551-570 PMID:2687720.

673 Buzsaki G (2015) Hippocampal sharp wave-ripple: A cognitive biomarker for episodic memory

674 and planning. Hippocampus 25:1073-1188. PMCID: PMC4648295 PMID:26135716.

675 Chen Z, Grosmark AD, Penagos H, Wilson MA (2016) Uncovering representations of sleep-

676 associated hippocampal ensemble spike activity. Sci Rep 6:32193. PMCID:

677 PMCPMC5004124 PMID:27573200.

678 Cheng S, Frank LM (2008) New experiences enhance coordinated neural activity in the

679 hippocampus. Neuron 57:303-313. PMCID: PMC2244590 PMID:18215626.

680 Csicsvari J, Hirase H, Czurko A, Buzsaki G (1998) Reliability and state dependence of

681 pyramidal cell-interneuron synapses in the hippocampus: an ensemble approach in the

682 behaving rat. Neuron 21:179-189 PMID:9697862.

683 Diekelmann S, Born J (2010) The memory function of sleep. Nat Rev Neurosci 11:114-126

684 PMID:20046194.

685 Dragoi G, Tonegawa S (2011) Preplay of future place cell sequences by hippocampal cellular

686 assemblies. Nature 469:397-401 PMID:21179088.

687 Dragoi G, Tonegawa S (2013) Distinct preplay of multiple novel spatial experiences in the rat.

688 PNAS 110:9100-9105. PMCID: PMC3670374 PMID:23671088.

689 Dragoi G, Tonegawa S (2014) Selection of preconfigured cell assemblies for representation of

690 novel spatial experiences. Philos Trans R Soc Lond B Biol Sci 369:20120522. PMCID:

691 PMC3866444 PMID:24366134.

692 Dudai Y (2004) The neurobiology of consolidations, or, how stable is the engram? Annu Rev

693 Psychol 55:51-86 PMID:14744210.

694 Feng T, Silva D, Foster DJ (2015) Dissociation between the experience-dependent development

695 of hippocampal theta sequences and single-trial phase precession. J Neurosci 35:4890-4902.

696 PMCID: PMCPMC4389593 PMID:25810520.

697 Florian C, Vecsey CG, Halassa MM, Haydon PG, Abel T (2011) Astrocyte-derived adenosine

698 and A1 receptor activity contribute to sleep loss-induced deficits in hippocampal synaptic

699 plasticity and memory in mice. J Neurosci 31:6956-6962. PMCID: PMCPMC3140051

700 PMID:21562257.

701 Frank LM, Stanley GB, Brown EN (2004) Hippocampal plasticity across multiple days of

702 exposure to novel environments. J Neurosci 24:7681-7689 PMID:15342735.

703 Graves LA, Heller EA, Pack AI, Abel T (2003) Sleep deprivation selectively impairs memory

704 consolidation for contextual fear conditioning. Learn Mem 10:168-176. PMCID:

705 PMC202307 PMID:12773581.

706 Grosmark AD, Buzsaki G (2016) Diversity in neural firing dynamics supports both rigid and

707 learned hippocampal sequences. Science 351:1440-1443. PMCID: PMC4919122

708 PMID:27013730.

709 Grosmark AD, Long JD, Buzsáki G (2016) Recordings from hippocampal area CA1, PRE,

710 during and POST novel spatial learning. In: CRCNS.org.

711 Harris KD, Henze DA, Csicsvari J, Hirase H, Buzsaki G (2000) Accuracy of tetrode spike

712 separation as determined by simultaneous intracellular and extracellular measurements. J

713 Neurophysiol 84:401-414 PMID:10899214.

714 Havekes R, Abel T (2017) The tired hippocampus: the molecular impact of sleep deprivation on

715 hippocampal function. Curr Opin Neurobiol 44:13-19. PMCID: PMCPMC5511071

716 PMID:28242433.

717 Havekes R, Bruinenberg VM, Tudor JC, Ferri SL, Baumann A, Meerlo P, Abel T (2014)

718 Transiently increasing cAMP levels selectively in hippocampal excitatory neurons during

719 sleep deprivation prevents memory deficits caused by sleep loss. J Neurosci 34:15715-15721.

720 PMCID: PMC4236401 PMID:25411499.

721 Hazan L, Zugaro M, Buzsaki G (2006) Klusters, NeuroScope, NDManager: a free software suite

722 for neurophysiological data processing and visualization. J Neurosci Methods 155:207-216

723 PMID:16580733.

724 Hirase H, Leinekugel X, Czurko A, Csicsvari J, Buzsaki G (2001) Firing rates of hippocampal

725 neurons are preserved during subsequent sleep episodes and modified by novel awake

726 experience. Proc Natl Acad Sci U S A 98:9386-9390 PMID:11470910.

727 Hoeffer CA, Klann E (2010) mTOR signaling: at the crossroads of plasticity, memory and

728 disease. Trends Neurosci 33:67-75. PMCID: PMCPMC2821969 PMID:19963289.

729 Ji D, Wilson MA (2007) Coordinated memory replay in the visual cortex and hippocampus

730 during sleep. Nat Neurosci 10:100-107 PMID:17173043.

731 Johnson LA, Euston DR, Tatsuno M, McNaughton BL (2010) Stored-trace reactivation in rat

732 prefrontal cortex is correlated with down-to-up state fluctuation density. J Neurosci 30:2650-

733 2661. PMCID: PMC2917239 PMID:20164349.

734 Karlsson MP, Frank LM (2008) Network dynamics underlying the formation of sparse,

735 informative representations in the hippocampus. J Neurosci 28:14271-14281. PMCID:

736 PMC2632980 PMID:19109508.

737 Kudrimoti HS, Barnes CA, McNaughton BL (1999) Reactivation of hippocampal cell

738 assemblies: effects of behavioral state, experience, and EEG dynamics. J Neurosci 19:4090-

739 4101 PMID:10234037.

740 Larkin MC, Lykken C, Tye LD, Wickelgren JG, Frank LM (2014) Hippocampal output area

741 CA1 broadcasts a generalized novelty signal during an object-place recognition task.

742 Hippocampus 24:773-783. PMCID: PMC4065199 PMID:24596296.

743 Lee AK, Wilson MA (2002) Memory of sequential experience in the hippocampus during slow

744 wave sleep. Neuron 36:1183-1194 PMID:12495631.

745 Leutgeb S, Leutgeb JK, Barnes CA, Moser EI, McNaughton BL, Moser MB (2005) Independent

746 codes for spatial and episodic memory in hippocampal neuronal ensembles. Science 309:619-

747 623 PMID:16040709.

748 Lopes-dos-Santos V, Ribeiro S, Tort AB (2013) Detecting cell assemblies in large neuronal

749 populations. J Neurosci Methods 220:149-166 PMID:23639919.

750 Lopes-dos-Santos V, Conde-Ocazionez S, Nicolelis MA, Ribeiro ST, Tort AB (2011) Neuronal

751 assembly detection and cell membership specification by principal component analysis. PloS

752 One 6:e20996. PMCID: PMC3115970 PMID:21698248.

753 Maboudi K, Ackermann E, Kemere C, Diba K (2018a) Temporal structure of hippocampal

754 activity during offline periods. In: Cosyne Conference. Denver, CO.

755 Maboudi K, Ackermann E, Pfeiffer BE, Foster DJ, Diba K, Kemere C (2018b) Uncovering

756 temporal structure in hippocampal output patterns. bioRxiv.

757 Maingret N, Girardeau G, Todorova R, Goutierre M, Zugaro M (2016) Hippocampo-cortical

758 coupling mediates memory consolidation during sleep. Nat Neurosci 19:959-964

759 PMID:27182818.

760 Marr D (1971) Simple memory: a theory for archicortex. Philos Trans R Soc Lond B Biol Sci

761 262:23-81 PMID:4399412.

762 McGaugh JL (2000) Memory--a century of consolidation. Science 287:248-251

763 PMID:10634773.

764 Miyawaki H, Diba K (2016) Regulation of Hippocampal Firing by Network Oscillations during

765 Sleep. Curr Biol 26:893-902. PMCID: PMC4821660 PMID:26972321.

766 Mizuseki K, Sirota A, Pastalkova E, Buzsaki G (2009) Theta oscillations provide temporal

767 windows for local circuit computation in the entorhinal-hippocampal loop. Neuron 64:267-

768 280. PMCID: PMC2771122 PMID:19874793.

769 Mizuseki K, Diba K, Pastalkova E, Buzsaki G (2011) Hippocampal CA1 pyramidal cells form

770 functionally distinct sublayers. Nat Neurosci 14:1174-1181. PMCID: PMC3164922

771 PMID:21822270.

772 Moita MA, Rosis S, Zhou Y, LeDoux JE, Blair HT (2004) Putting fear in its place: remapping of

773 hippocampal place cells during fear conditioning. J Neurosci 24:7015-7023 PMID:15295037.

774 Muller RU, Kubie JL (1987) The effects of changes in the environment on the spatial firing of

775 hippocampal complex-spike cells. J Neurosci 7:1951-1968 PMID:3612226.

776 Nadasdy Z, Hirase H, Czurko A, Csicsvari J, Buzsaki G (1999) Replay and time compression of

777 recurring spike sequences in the hippocampus. J Neurosci 19:9497-9507 PMID:10531452.

778 Norimoto H, Makino K, Gao M, Shikano Y, Okamoto K, Ishikawa T, Sasaki T, Hioki H,

779 Fujisawa S, Ikegaya Y (2018) Hippocampal ripples down-regulate synapses. Science

780 359:1524-1527 PMID:29439023.

781 Ognjanovski N, Schaeffer S, Wu J, Mofakham S, Maruyama D, Zochowski M, Aton SJ (2017)

782 Parvalbumin-expressing interneurons coordinate hippocampal network dynamics required for

783 memory consolidation. Nat Commun 8:15039. PMCID: PMCPMC5384212

784 PMID:28382952.

785 Palchykova S, Winsky-Sommerer R, Meerlo P, Durr R, Tobler I (2006) Sleep deprivation

786 impairs object recognition in mice. Neurobiol Learn Mem 85:263-271 PMID:16423541.

787 Pastalkova E, Itskov V, Amarasingham A, Buzsaki G (2008) Internally generated cell assembly

788 sequences in the rat hippocampus. Science 321:1322-1327. PMCID: PMC2570043

789 PMID:18772431.

790 Peyrache A, Khamassi M, Benchenane K, Wiener SI, Battaglia FP (2009) Replay of rule-

791 learning related neural patterns in the prefrontal cortex during sleep. Nat Neurosci 12:919-

792 926 PMID:19483687.

793 Prince TM, Wimmer M, Choi J, Havekes R, Aton S, Abel T (2014) Sleep deprivation during a

794 specific 3-hour time window post-training impairs hippocampal synaptic plasticity and

795 memory. Neurobiol Learn Mem 109:122-130. PMCID: PMC3966473 PMID:24380868.

796 Rasch B, Born J (2013) About sleep's role in memory. Physiol Rev 93:681-766 PMID:23589831.

797 Ribeiro S, Gervasoni D, Soares ES, Zhou Y, Lin SC, Pantoja J, Lavine M, Nicolelis MA (2004)

798 Long-lasting novelty-induced neuronal reverberation during slow-wave sleep in multiple

799 forebrain areas. PLoS Biol 2:E24. PMCID: PMC314474 PMID:14737198.

800 Schmitzer-Torbert N, Jackson J, Henze D, Harris K, Redish AD (2005) Quantitative measures of

801 cluster quality for use in extracellular recordings. Neuroscience 131:1-11 PMID:15680687.

802 Skaggs WE, McNaughton BL (1996) Replay of neuronal firing sequences in rat hippocampus

803 during sleep following spatial experience. Science 271:1870-1873 PMID:8596957.

804 Sutherland RJ, Sparks FT, Lehmann H (2010) Hippocampus and retrograde amnesia in the rat

805 model: a modest proposal for the situation of systems consolidation. Neuropsychologia

806 48:2357-2369. PMCID: PMC2900526 PMID:20430043.

807 Takeuchi T, Duszkiewicz AJ, Morris RG (2014) The synaptic plasticity and memory hypothesis:

808 encoding, storage and persistence. Philos Trans R Soc Lond B Biol Sci 369:20130288.

809 PMCID: PMCPMC3843897 PMID:24298167.

810 Takeuchi T, Duszkiewicz AJ, Sonneborn A, Spooner PA, Yamasaki M, Watanabe M, Smith CC,

811 Fernandez G, Deisseroth K, Greene RW, Morris RG (2016) Locus coeruleus and

812 dopaminergic consolidation of everyday memory. Nature 537:357-362. PMCID:

813 PMCPMC5161591 PMID:27602521.

814 Tatsuno M, Lipa P, McNaughton BL (2006) Methodological considerations on the use of

815 template matching to study long-lasting memory trace replay. J Neurosci 26:10727-10742

816 PMID:17050712.

817 Tononi G, Cirelli C (2014) Sleep and the price of plasticity: from synaptic and cellular

818 homeostasis to memory consolidation and integration. Neuron 81:12-34 PMID:24411729.

819 Trouche S, Perestenko PV, van de Ven GM, Bratley CT, McNamara CG, Campo-Urriza N,

820 Black SL, Reijmers LG, Dupret D (2016) Recoding a cocaine-place memory engram to a

821 neutral engram in the hippocampus. Nat Neurosci 19:564-567. PMCID: PMCPMC4817230

822 PMID:26900924.

823 Tudor JC, Davis EJ, Peixoto L, Wimmer ME, van Tilborg E, Park AJ, Poplawski SG, Chung

824 CW, Havekes R, Huang J, Gatti E, Pierre P, Abel T (2016) Sleep deprivation impairs

825 memory by attenuating mTORC1-dependent protein synthesis. Science signaling 9:ra41.

826 PMCID: PMC4890572 PMID:27117251.

827 van de Ven GM, Trouche S, McNamara CG, Allen K, Dupret D (2016) Hippocampal Offline

828 Reactivation Consolidates Recently Formed Cell Assembly Patterns during Sharp Wave-

829 Ripples. Neuron 92:968-974. PMCID: PMCPMC5158132 PMID:27840002.

830 Vecsey CG, Peixoto L, Choi JH, Wimmer M, Jaganath D, Hernandez PJ, Blackwell J, Meda K,

831 Park AJ, Hannenhalli S, Abel T (2012) Genomic analysis of sleep deprivation reveals

832 translational regulation in the hippocampus. Physiol Genomics 44:981-991. PMCID:

833 PMCPMC3472468 PMID:22930738.

834 Vecsey CG, Baillie GS, Jaganath D, Havekes R, Daniels A, Wimmer M, Huang T, Brown KM,

835 Li XY, Descalzi G, Kim SS, Chen T, Shang YZ, Zhuo M, Houslay MD, Abel T (2009) Sleep

836 deprivation impairs cAMP signalling in the hippocampus. Nature 461:1122-1125. PMCID:

837 PMC2783639 PMID:19847264.

838 Wang ME, Wann EG, Yuan RK, Ramos Alvarez MM, Stead SM, Muzzio IA (2012) Long-term

839 stabilization of place cell remapping produced by a fearful experience. J Neurosci 32:15802-

840 15814. PMCID: PMCPMC3547638 PMID:23136419.

841 Wikenheiser AM, Redish AD (2013) The balance of forward and backward hippocampal

842 sequences shifts across behavioral states. Hippocampus 23:22-29. PMCID:

843 PMCPMC3774294 PMID:22736562.

844 Wilson MA, McNaughton BL (1994) Reactivation of hippocampal ensemble memories during

845 sleep. Science 265:676-679 PMID:8036517.

846

847

848

849

850

851

852

853

854