megaparsec-9.7.0: Monadic parser combinators

Copyright | © 2015–present Megaparsec contributors
License | FreeBSD
Maintainer | Mark Karpov <markkarpov92@gmail.com>
Stability | experimental
Portability | portable
Safe Haskell | Safe
Language | Haskell2010

Text.Megaparsec.Stream

Description

Megaparsec's input stream facilities.

You probably do not want to import this module directly because Text.Megaparsec re-exports it anyway.

Since: 6.0.0

Synopsis

- class (Ord (Token s), Ord (Tokens s)) => Stream s where
  - type Token s
  - type Tokens s
  - tokenToChunk :: Proxy s -> Token s -> Tokens s
  - tokensToChunk :: Proxy s -> [Token s] -> Tokens s
  - chunkToTokens :: Proxy s -> Tokens s -> [Token s]
  - chunkLength :: Proxy s -> Tokens s -> Int
  - chunkEmpty :: Proxy s -> Tokens s -> Bool
  - take1_ :: s -> Maybe (Token s, s)
  - takeN_ :: Int -> s -> Maybe (Tokens s, s)
  - takeWhile_ :: (Token s -> Bool) -> s -> (Tokens s, s)
- newtype ShareInput a = ShareInput {
  - unShareInput :: a
  }
- newtype NoShareInput a = NoShareInput {
  - unNoShareInput :: a
  }
- class Stream s => VisualStream s where
  - showTokens :: Proxy s -> NonEmpty (Token s) -> String
  - tokensLength :: Proxy s -> NonEmpty (Token s) -> Int
- class Stream s => TraversableStream s where
  - reachOffset :: Int -> PosState s -> (Maybe String, PosState s)
  - reachOffsetNoLine :: Int -> PosState s -> PosState s

# Documentation

class (Ord (Token s), Ord (Tokens s)) => Stream s where #

Type class for inputs that can be consumed by the library.

Note that the `Stream` instances for `Text` and `ByteString` (strict and lazy) default to "input sharing" (see `ShareInput`, `NoShareInput`). We plan to move away from input sharing in a future major release; if you want to retain the current behaviour and are concerned with maximum performance you should consider using the `ShareInput` wrapper explicitly.

Note: before the version 9.0.0 the class included the methods from `VisualStream` and `TraversableStream`.

Minimal complete definition

tokensToChunk, chunkToTokens, chunkLength, take1_, takeN_, takeWhile_

Associated Types

type Token s #

Type of token in the stream.

type Tokens s #

Type of “chunk” of the stream.

Methods

tokenToChunk :: Proxy s -> Token s -> Tokens s #

Lift a single token to chunk of the stream. The default implementation is:

    tokenToChunk pxy = tokensToChunk pxy . pure

However for some types of stream there may be a more efficient way to lift.

tokensToChunk :: Proxy s -> [Token s] -> Tokens s #

The first method that establishes isomorphism between list of tokens and chunk of the stream. Valid implementation should satisfy:

    chunkToTokens pxy (tokensToChunk pxy ts) == ts

chunkToTokens :: Proxy s -> Tokens s -> [Token s] #

The second method that establishes isomorphism between list of tokens and chunk of the stream. Valid implementation should satisfy:

    tokensToChunk pxy (chunkToTokens pxy chunk) == chunk

chunkLength :: Proxy s -> Tokens s -> Int #

Return length of a chunk of the stream.

chunkEmpty :: Proxy s -> Tokens s -> Bool #

Check if a chunk of the stream is empty. The default implementation is in terms of the more general `chunkLength`:

    chunkEmpty pxy ts = chunkLength pxy ts <= 0

However for many streams there may be a more efficient implementation.

take1_ :: s -> Maybe (Token s, s) #

Extract a single token form the stream. Return `Nothing` if the stream is empty.

takeN_ :: Int -> s -> Maybe (Tokens s, s) #

``takeN_` n s` should try to extract a chunk of length `n`, or if the stream is too short, the rest of the stream. Valid implementation should follow the rules:

- If the requested length `n` is 0 (or less), `Nothing` should never be returned, instead ``Just` ("", s)` should be returned, where `""` stands for the empty chunk, and `s` is the original stream (second argument).
- If the requested length is greater than 0 and the stream is empty, `Nothing` should be returned indicating end of input.
- In other cases, take chunk of length `n` (or shorter if the stream is not long enough) from the input stream and return the chunk along with the rest of the stream.

takeWhile_ :: (Token s -> Bool) -> s -> (Tokens s, s) #

Extract chunk of the stream taking tokens while the supplied predicate returns `True`. Return the chunk and the rest of the stream.

For many types of streams, the method allows for significant performance improvements, although it is not strictly necessary from conceptual point of view.

#### Instances

Instances details

Stream ByteString # |
Instance details Defined in Text.Megaparsec.Stream Associated Types
type Token ByteString |
Instance details Defined in Text.Megaparsec.Stream type Token ByteString = Token (ShareInput ByteString)
type Tokens ByteString |
Instance details Defined in Text.Megaparsec.Stream type Tokens ByteString = Tokens (ShareInput ByteString)
Methods tokenToChunk :: Proxy ByteString -> Token ByteString -> Tokens ByteString # tokensToChunk :: Proxy ByteString -> [Token ByteString] -> Tokens ByteString # chunkToTokens :: Proxy ByteString -> Tokens ByteString -> [Token ByteString] # chunkLength :: Proxy ByteString -> Tokens ByteString -> Int # chunkEmpty :: Proxy ByteString -> Tokens ByteString -> Bool # take1_ :: ByteString -> Maybe (Token ByteString, ByteString) # takeN_ :: Int -> ByteString -> Maybe (Tokens ByteString, ByteString) # takeWhile_ :: (Token ByteString -> Bool) -> ByteString -> (Tokens ByteString, ByteString) #
Stream ByteString # |
Instance details Defined in Text.Megaparsec.Stream Associated Types
type Token ByteString |
Instance details Defined in Text.Megaparsec.Stream type Token ByteString = Token (ShareInput ByteString)
type Tokens ByteString |
Instance details Defined in Text.Megaparsec.Stream type Tokens ByteString = Tokens (ShareInput ByteString)
Methods tokenToChunk :: Proxy ByteString -> Token ByteString -> Tokens ByteString # tokensToChunk :: Proxy ByteString -> [Token ByteString] -> Tokens ByteString # chunkToTokens :: Proxy ByteString -> Tokens ByteString -> [Token ByteString] # chunkLength :: Proxy ByteString -> Tokens ByteString -> Int # chunkEmpty :: Proxy ByteString -> Tokens ByteString -> Bool # take1_ :: ByteString -> Maybe (Token ByteString, ByteString) # takeN_ :: Int -> ByteString -> Maybe (Tokens ByteString, ByteString) # takeWhile_ :: (Token ByteString -> Bool) -> ByteString -> (Tokens ByteString, ByteString) #
Stream Text # |
Instance details Defined in Text.Megaparsec.Stream Associated Types
type Token Text |
Instance details Defined in Text.Megaparsec.Stream type Token Text = Token (ShareInput Text)
type Tokens Text |
Instance details Defined in Text.Megaparsec.Stream type Tokens Text = Tokens (ShareInput Text)
Methods tokenToChunk :: Proxy Text -> Token Text -> Tokens Text # tokensToChunk :: Proxy Text -> [Token Text] -> Tokens Text # chunkToTokens :: Proxy Text -> Tokens Text -> [Token Text] # chunkLength :: Proxy Text -> Tokens Text -> Int # chunkEmpty :: Proxy Text -> Tokens Text -> Bool # take1_ :: Text -> Maybe (Token Text, Text) # takeN_ :: Int -> Text -> Maybe (Tokens Text, Text) # takeWhile_ :: (Token Text -> Bool) -> Text -> (Tokens Text, Text) #
Stream Text # |
Instance details Defined in Text.Megaparsec.Stream Associated Types
type Token Text |
Instance details Defined in Text.Megaparsec.Stream type Token Text = Token (ShareInput Text)
type Tokens Text |
Instance details Defined in Text.Megaparsec.Stream type Tokens Text = Tokens (ShareInput Text)
Methods tokenToChunk :: Proxy Text -> Token Text -> Tokens Text # tokensToChunk :: Proxy Text -> [Token Text] -> Tokens Text # chunkToTokens :: Proxy Text -> Tokens Text -> [Token Text] # chunkLength :: Proxy Text -> Tokens Text -> Int # chunkEmpty :: Proxy Text -> Tokens Text -> Bool # take1_ :: Text -> Maybe (Token Text, Text) # takeN_ :: Int -> Text -> Maybe (Tokens Text, Text) # takeWhile_ :: (Token Text -> Bool) -> Text -> (Tokens Text, Text) #
Ord a => Stream (Seq a) # | Since: 9.0.0
Instance details Defined in Text.Megaparsec.Stream Associated Types
type Token (Seq a) |
Instance details Defined in Text.Megaparsec.Stream type Token (Seq a) = a
type Tokens (Seq a) |
Instance details Defined in Text.Megaparsec.Stream type Tokens (Seq a) = Seq a
Methods tokenToChunk :: Proxy (Seq a) -> Token (Seq a) -> Tokens (Seq a) # tokensToChunk :: Proxy (Seq a) -> [Token (Seq a)] -> Tokens (Seq a) # chunkToTokens :: Proxy (Seq a) -> Tokens (Seq a) -> [Token (Seq a)] # chunkLength :: Proxy (Seq a) -> Tokens (Seq a) -> Int # chunkEmpty :: Proxy (Seq a) -> Tokens (Seq a) -> Bool # take1_ :: Seq a -> Maybe (Token (Seq a), Seq a) # takeN_ :: Int -> Seq a -> Maybe (Tokens (Seq a), Seq a) # takeWhile_ :: (Token (Seq a) -> Bool) -> Seq a -> (Tokens (Seq a), Seq a) #
Stream (NoShareInput ByteString) # |
Instance details Defined in Text.Megaparsec.Stream Associated Types
type Token (NoShareInput ByteString) |
Instance details Defined in Text.Megaparsec.Stream type Token (NoShareInput ByteString) = Word8
type Tokens (NoShareInput ByteString) |
Instance details Defined in Text.Megaparsec.Stream type Tokens (NoShareInput ByteString) = ByteString
Methods tokenToChunk :: Proxy (NoShareInput ByteString) -> Token (NoShareInput ByteString) -> Tokens (NoShareInput ByteString) # tokensToChunk :: Proxy (NoShareInput ByteString) -> [Token (NoShareInput ByteString)] -> Tokens (NoShareInput ByteString) # chunkToTokens :: Proxy (NoShareInput ByteString) -> Tokens (NoShareInput ByteString) -> [Token (NoShareInput ByteString)] # chunkLength :: Proxy (NoShareInput ByteString) -> Tokens (NoShareInput ByteString) -> Int # chunkEmpty :: Proxy (NoShareInput ByteString) -> Tokens (NoShareInput ByteString) -> Bool # take1_ :: NoShareInput ByteString -> Maybe (Token (NoShareInput ByteString), NoShareInput ByteString) # takeN_ :: Int -> NoShareInput ByteString -> Maybe (Tokens (NoShareInput ByteString), NoShareInput ByteString) # takeWhile_ :: (Token (NoShareInput ByteString) -> Bool) -> NoShareInput ByteString -> (Tokens (NoShareInput ByteString), NoShareInput ByteString) #
Stream (NoShareInput ByteString) # |
Instance details Defined in Text.Megaparsec.Stream Associated Types
type Token (NoShareInput ByteString) |
Instance details Defined in Text.Megaparsec.Stream type Token (NoShareInput ByteString) = Word8
type Tokens (NoShareInput ByteString) |
Instance details Defined in Text.Megaparsec.Stream type Tokens (NoShareInput ByteString) = ByteString
Methods tokenToChunk :: Proxy (NoShareInput ByteString) -> Token (NoShareInput ByteString) -> Tokens (NoShareInput ByteString) # tokensToChunk :: Proxy (NoShareInput ByteString) -> [Token (NoShareInput ByteString)] -> Tokens (NoShareInput ByteString) # chunkToTokens :: Proxy (NoShareInput ByteString) -> Tokens (NoShareInput ByteString) -> [Token (NoShareInput ByteString)] # chunkLength :: Proxy (NoShareInput ByteString) -> Tokens (NoShareInput ByteString) -> Int # chunkEmpty :: Proxy (NoShareInput ByteString) -> Tokens (NoShareInput ByteString) -> Bool # take1_ :: NoShareInput ByteString -> Maybe (Token (NoShareInput ByteString), NoShareInput ByteString) # takeN_ :: Int -> NoShareInput ByteString -> Maybe (Tokens (NoShareInput ByteString), NoShareInput ByteString) # takeWhile_ :: (Token (NoShareInput ByteString) -> Bool) -> NoShareInput ByteString -> (Tokens (NoShareInput ByteString), NoShareInput ByteString) #
Stream (NoShareInput Text) # |
Instance details Defined in Text.Megaparsec.Stream Associated Types
type Token (NoShareInput Text) |
Instance details Defined in Text.Megaparsec.Stream type Token (NoShareInput Text) = Char
type Tokens (NoShareInput Text) |
Instance details Defined in Text.Megaparsec.Stream type Tokens (NoShareInput Text) = Text
Methods tokenToChunk :: Proxy (NoShareInput Text) -> Token (NoShareInput Text) -> Tokens (NoShareInput Text) # tokensToChunk :: Proxy (NoShareInput Text) -> [Token (NoShareInput Text)] -> Tokens (NoShareInput Text) # chunkToTokens :: Proxy (NoShareInput Text) -> Tokens (NoShareInput Text) -> [Token (NoShareInput Text)] # chunkLength :: Proxy (NoShareInput Text) -> Tokens (NoShareInput Text) -> Int # chunkEmpty :: Proxy (NoShareInput Text) -> Tokens (NoShareInput Text) -> Bool # take1_ :: NoShareInput Text -> Maybe (Token (NoShareInput Text), NoShareInput Text) # takeN_ :: Int -> NoShareInput Text -> Maybe (Tokens (NoShareInput Text), NoShareInput Text) # takeWhile_ :: (Token (NoShareInput Text) -> Bool) -> NoShareInput Text -> (Tokens (NoShareInput Text), NoShareInput Text) #
Stream (NoShareInput Text) # |
Instance details Defined in Text.Megaparsec.Stream Associated Types
type Token (NoShareInput Text) |
Instance details Defined in Text.Megaparsec.Stream type Token (NoShareInput Text) = Char
type Tokens (NoShareInput Text) |
Instance details Defined in Text.Megaparsec.Stream type Tokens (NoShareInput Text) = Text
Methods tokenToChunk :: Proxy (NoShareInput Text) -> Token (NoShareInput Text) -> Tokens (NoShareInput Text) # tokensToChunk :: Proxy (NoShareInput Text) -> [Token (NoShareInput Text)] -> Tokens (NoShareInput Text) # chunkToTokens :: Proxy (NoShareInput Text) -> Tokens (NoShareInput Text) -> [Token (NoShareInput Text)] # chunkLength :: Proxy (NoShareInput Text) -> Tokens (NoShareInput Text) -> Int # chunkEmpty :: Proxy (NoShareInput Text) -> Tokens (NoShareInput Text) -> Bool # take1_ :: NoShareInput Text -> Maybe (Token (NoShareInput Text), NoShareInput Text) # takeN_ :: Int -> NoShareInput Text -> Maybe (Tokens (NoShareInput Text), NoShareInput Text) # takeWhile_ :: (Token (NoShareInput Text) -> Bool) -> NoShareInput Text -> (Tokens (NoShareInput Text), NoShareInput Text) #
Stream (ShareInput ByteString) # |
Instance details Defined in Text.Megaparsec.Stream Associated Types
type Token (ShareInput ByteString) |
Instance details Defined in Text.Megaparsec.Stream type Token (ShareInput ByteString) = Word8
type Tokens (ShareInput ByteString) |
Instance details Defined in Text.Megaparsec.Stream type Tokens (ShareInput ByteString) = ByteString
Methods tokenToChunk :: Proxy (ShareInput ByteString) -> Token (ShareInput ByteString) -> Tokens (ShareInput ByteString) # tokensToChunk :: Proxy (ShareInput ByteString) -> [Token (ShareInput ByteString)] -> Tokens (ShareInput ByteString) # chunkToTokens :: Proxy (ShareInput ByteString) -> Tokens (ShareInput ByteString) -> [Token (ShareInput ByteString)] # chunkLength :: Proxy (ShareInput ByteString) -> Tokens (ShareInput ByteString) -> Int # chunkEmpty :: Proxy (ShareInput ByteString) -> Tokens (ShareInput ByteString) -> Bool # take1_ :: ShareInput ByteString -> Maybe (Token (ShareInput ByteString), ShareInput ByteString) # takeN_ :: Int -> ShareInput ByteString -> Maybe (Tokens (ShareInput ByteString), ShareInput ByteString) # takeWhile_ :: (Token (ShareInput ByteString) -> Bool) -> ShareInput ByteString -> (Tokens (ShareInput ByteString), ShareInput ByteString) #
Stream (ShareInput ByteString) # |
Instance details Defined in Text.Megaparsec.Stream Associated Types
type Token (ShareInput ByteString) |
Instance details Defined in Text.Megaparsec.Stream type Token (ShareInput ByteString) = Word8
type Tokens (ShareInput ByteString) |
Instance details Defined in Text.Megaparsec.Stream type Tokens (ShareInput ByteString) = ByteString
Methods tokenToChunk :: Proxy (ShareInput ByteString) -> Token (ShareInput ByteString) -> Tokens (ShareInput ByteString) # tokensToChunk :: Proxy (ShareInput ByteString) -> [Token (ShareInput ByteString)] -> Tokens (ShareInput ByteString) # chunkToTokens :: Proxy (ShareInput ByteString) -> Tokens (ShareInput ByteString) -> [Token (ShareInput ByteString)] # chunkLength :: Proxy (ShareInput ByteString) -> Tokens (ShareInput ByteString) -> Int # chunkEmpty :: Proxy (ShareInput ByteString) -> Tokens (ShareInput ByteString) -> Bool # take1_ :: ShareInput ByteString -> Maybe (Token (ShareInput ByteString), ShareInput ByteString) # takeN_ :: Int -> ShareInput ByteString -> Maybe (Tokens (ShareInput ByteString), ShareInput ByteString) # takeWhile_ :: (Token (ShareInput ByteString) -> Bool) -> ShareInput ByteString -> (Tokens (ShareInput ByteString), ShareInput ByteString) #
Stream (ShareInput Text) # |
Instance details Defined in Text.Megaparsec.Stream Associated Types
type Token (ShareInput Text) |
Instance details Defined in Text.Megaparsec.Stream type Token (ShareInput Text) = Char
type Tokens (ShareInput Text) |
Instance details Defined in Text.Megaparsec.Stream type Tokens (ShareInput Text) = Text
Methods tokenToChunk :: Proxy (ShareInput Text) -> Token (ShareInput Text) -> Tokens (ShareInput Text) # tokensToChunk :: Proxy (ShareInput Text) -> [Token (ShareInput Text)] -> Tokens (ShareInput Text) # chunkToTokens :: Proxy (ShareInput Text) -> Tokens (ShareInput Text) -> [Token (ShareInput Text)] # chunkLength :: Proxy (ShareInput Text) -> Tokens (ShareInput Text) -> Int # chunkEmpty :: Proxy (ShareInput Text) -> Tokens (ShareInput Text) -> Bool # take1_ :: ShareInput Text -> Maybe (Token (ShareInput Text), ShareInput Text) # takeN_ :: Int -> ShareInput Text -> Maybe (Tokens (ShareInput Text), ShareInput Text) # takeWhile_ :: (Token (ShareInput Text) -> Bool) -> ShareInput Text -> (Tokens (ShareInput Text), ShareInput Text) #
Stream (ShareInput Text) # |
Instance details Defined in Text.Megaparsec.Stream Associated Types
type Token (ShareInput Text) |
Instance details Defined in Text.Megaparsec.Stream type Token (ShareInput Text) = Char
type Tokens (ShareInput Text) |
Instance details Defined in Text.Megaparsec.Stream type Tokens (ShareInput Text) = Text
Methods tokenToChunk :: Proxy (ShareInput Text) -> Token (ShareInput Text) -> Tokens (ShareInput Text) # tokensToChunk :: Proxy (ShareInput Text) -> [Token (ShareInput Text)] -> Tokens (ShareInput Text) # chunkToTokens :: Proxy (ShareInput Text) -> Tokens (ShareInput Text) -> [Token (ShareInput Text)] # chunkLength :: Proxy (ShareInput Text) -> Tokens (ShareInput Text) -> Int # chunkEmpty :: Proxy (ShareInput Text) -> Tokens (ShareInput Text) -> Bool # take1_ :: ShareInput Text -> Maybe (Token (ShareInput Text), ShareInput Text) # takeN_ :: Int -> ShareInput Text -> Maybe (Tokens (ShareInput Text), ShareInput Text) # takeWhile_ :: (Token (ShareInput Text) -> Bool) -> ShareInput Text -> (Tokens (ShareInput Text), ShareInput Text) #
Ord a => Stream [a] # | Since: 9.0.0
Instance details Defined in Text.Megaparsec.Stream Associated Types
type Token [a] |
Instance details Defined in Text.Megaparsec.Stream type Token [a] = a
type Tokens [a] |
Instance details Defined in Text.Megaparsec.Stream type Tokens [a] = [a]
Methods tokenToChunk :: Proxy [a] -> Token [a] -> Tokens [a] # tokensToChunk :: Proxy [a] -> [Token [a]] -> Tokens [a] # chunkToTokens :: Proxy [a] -> Tokens [a] -> [Token [a]] # chunkLength :: Proxy [a] -> Tokens [a] -> Int # chunkEmpty :: Proxy [a] -> Tokens [a] -> Bool # take1_ :: [a] -> Maybe (Token [a], [a]) # takeN_ :: Int -> [a] -> Maybe (Tokens [a], [a]) # takeWhile_ :: (Token [a] -> Bool) -> [a] -> (Tokens [a], [a]) #

newtype ShareInput a #

This wrapper selects the input-sharing `Stream` implementation for `Text` (`Text`) and `ByteString` (`ByteString`). By input sharing we mean that our parsers will use slices whenever possible to avoid having to copy parts of the input. See also the documentation of `split`.

Note that using slices is in general faster than copying; on the other hand it also has the potential for causing surprising memory leaks: if any slice of the input survives in the output, holding on to the output will force the entire input `Text`/`ByteString` to stay in memory! Even when using lazy `Text`/`ByteString` we will hold on to whole chunks at a time leading to to significantly worse memory residency in some cases.

See `NoShareInput` for a somewhat slower implementation that avoids this memory leak scenario.

Since: 9.3.0

Constructors

ShareInput |
Fields unShareInput :: a

#### Instances

Instances details

Stream (ShareInput ByteString) # |
Instance details Defined in Text.Megaparsec.Stream Associated Types
type Token (ShareInput ByteString) |
Instance details Defined in Text.Megaparsec.Stream type Token (ShareInput ByteString) = Word8
type Tokens (ShareInput ByteString) |
Instance details Defined in Text.Megaparsec.Stream type Tokens (ShareInput ByteString) = ByteString
Methods tokenToChunk :: Proxy (ShareInput ByteString) -> Token (ShareInput ByteString) -> Tokens (ShareInput ByteString) # tokensToChunk :: Proxy (ShareInput ByteString) -> [Token (ShareInput ByteString)] -> Tokens (ShareInput ByteString) # chunkToTokens :: Proxy (ShareInput ByteString) -> Tokens (ShareInput ByteString) -> [Token (ShareInput ByteString)] # chunkLength :: Proxy (ShareInput ByteString) -> Tokens (ShareInput ByteString) -> Int # chunkEmpty :: Proxy (ShareInput ByteString) -> Tokens (ShareInput ByteString) -> Bool # take1_ :: ShareInput ByteString -> Maybe (Token (ShareInput ByteString), ShareInput ByteString) # takeN_ :: Int -> ShareInput ByteString -> Maybe (Tokens (ShareInput ByteString), ShareInput ByteString) # takeWhile_ :: (Token (ShareInput ByteString) -> Bool) -> ShareInput ByteString -> (Tokens (ShareInput ByteString), ShareInput ByteString) #
Stream (ShareInput ByteString) # |
Instance details Defined in Text.Megaparsec.Stream Associated Types
type Token (ShareInput ByteString) |
Instance details Defined in Text.Megaparsec.Stream type Token (ShareInput ByteString) = Word8
type Tokens (ShareInput ByteString) |
Instance details Defined in Text.Megaparsec.Stream type Tokens (ShareInput ByteString) = ByteString
Methods tokenToChunk :: Proxy (ShareInput ByteString) -> Token (ShareInput ByteString) -> Tokens (ShareInput ByteString) # tokensToChunk :: Proxy (ShareInput ByteString) -> [Token (ShareInput ByteString)] -> Tokens (ShareInput ByteString) # chunkToTokens :: Proxy (ShareInput ByteString) -> Tokens (ShareInput ByteString) -> [Token (ShareInput ByteString)] # chunkLength :: Proxy (ShareInput ByteString) -> Tokens (ShareInput ByteString) -> Int # chunkEmpty :: Proxy (ShareInput ByteString) -> Tokens (ShareInput ByteString) -> Bool # take1_ :: ShareInput ByteString -> Maybe (Token (ShareInput ByteString), ShareInput ByteString) # takeN_ :: Int -> ShareInput ByteString -> Maybe (Tokens (ShareInput ByteString), ShareInput ByteString) # takeWhile_ :: (Token (ShareInput ByteString) -> Bool) -> ShareInput ByteString -> (Tokens (ShareInput ByteString), ShareInput ByteString) #
Stream (ShareInput Text) # |
Instance details Defined in Text.Megaparsec.Stream Associated Types
type Token (ShareInput Text) |
Instance details Defined in Text.Megaparsec.Stream type Token (ShareInput Text) = Char
type Tokens (ShareInput Text) |
Instance details Defined in Text.Megaparsec.Stream type Tokens (ShareInput Text) = Text
Methods tokenToChunk :: Proxy (ShareInput Text) -> Token (ShareInput Text) -> Tokens (ShareInput Text) # tokensToChunk :: Proxy (ShareInput Text) -> [Token (ShareInput Text)] -> Tokens (ShareInput Text) # chunkToTokens :: Proxy (ShareInput Text) -> Tokens (ShareInput Text) -> [Token (ShareInput Text)] # chunkLength :: Proxy (ShareInput Text) -> Tokens (ShareInput Text) -> Int # chunkEmpty :: Proxy (ShareInput Text) -> Tokens (ShareInput Text) -> Bool # take1_ :: ShareInput Text -> Maybe (Token (ShareInput Text), ShareInput Text) # takeN_ :: Int -> ShareInput Text -> Maybe (Tokens (ShareInput Text), ShareInput Text) # takeWhile_ :: (Token (ShareInput Text) -> Bool) -> ShareInput Text -> (Tokens (ShareInput Text), ShareInput Text) #
Stream (ShareInput Text) # |
Instance details Defined in Text.Megaparsec.Stream Associated Types
type Token (ShareInput Text) |
Instance details Defined in Text.Megaparsec.Stream type Token (ShareInput Text) = Char
type Tokens (ShareInput Text) |
Instance details Defined in Text.Megaparsec.Stream type Tokens (ShareInput Text) = Text
Methods tokenToChunk :: Proxy (ShareInput Text) -> Token (ShareInput Text) -> Tokens (ShareInput Text) # tokensToChunk :: Proxy (ShareInput Text) -> [Token (ShareInput Text)] -> Tokens (ShareInput Text) # chunkToTokens :: Proxy (ShareInput Text) -> Tokens (ShareInput Text) -> [Token (ShareInput Text)] # chunkLength :: Proxy (ShareInput Text) -> Tokens (ShareInput Text) -> Int # chunkEmpty :: Proxy (ShareInput Text) -> Tokens (ShareInput Text) -> Bool # take1_ :: ShareInput Text -> Maybe (Token (ShareInput Text), ShareInput Text) # takeN_ :: Int -> ShareInput Text -> Maybe (Tokens (ShareInput Text), ShareInput Text) # takeWhile_ :: (Token (ShareInput Text) -> Bool) -> ShareInput Text -> (Tokens (ShareInput Text), ShareInput Text) #
type Token (ShareInput ByteString) # |
Instance details Defined in Text.Megaparsec.Stream type Token (ShareInput ByteString) = Word8
type Token (ShareInput ByteString) # |
Instance details Defined in Text.Megaparsec.Stream type Token (ShareInput ByteString) = Word8
type Token (ShareInput Text) # |
Instance details Defined in Text.Megaparsec.Stream type Token (ShareInput Text) = Char
type Token (ShareInput Text) # |
Instance details Defined in Text.Megaparsec.Stream type Token (ShareInput Text) = Char
type Tokens (ShareInput ByteString) # |
Instance details Defined in Text.Megaparsec.Stream type Tokens (ShareInput ByteString) = ByteString
type Tokens (ShareInput ByteString) # |
Instance details Defined in Text.Megaparsec.Stream type Tokens (ShareInput ByteString) = ByteString
type Tokens (ShareInput Text) # |
Instance details Defined in Text.Megaparsec.Stream type Tokens (ShareInput Text) = Text
type Tokens (ShareInput Text) # |
Instance details Defined in Text.Megaparsec.Stream type Tokens (ShareInput Text) = Text

newtype NoShareInput a #

This wrapper selects the no-input-sharing `Stream` implementation for `Text` (`Text`) and `ByteString` (`ByteString`). This means that our parsers will create independent copies rather than using slices of the input. See also the documentation of `copy`.

More importantly, any parser output will be independent of the input, and holding on to parts of the output will never prevent the input from being garbage collected.

For maximum performance you might consider using `ShareInput` instead, but beware of its pitfalls!

Since: 9.3.0

Constructors

NoShareInput |
Fields unNoShareInput :: a

#### Instances

Instances details

Stream (NoShareInput ByteString) # |
Instance details Defined in Text.Megaparsec.Stream Associated Types
type Token (NoShareInput ByteString) |
Instance details Defined in Text.Megaparsec.Stream type Token (NoShareInput ByteString) = Word8
type Tokens (NoShareInput ByteString) |
Instance details Defined in Text.Megaparsec.Stream type Tokens (NoShareInput ByteString) = ByteString
Methods tokenToChunk :: Proxy (NoShareInput ByteString) -> Token (NoShareInput ByteString) -> Tokens (NoShareInput ByteString) # tokensToChunk :: Proxy (NoShareInput ByteString) -> [Token (NoShareInput ByteString)] -> Tokens (NoShareInput ByteString) # chunkToTokens :: Proxy (NoShareInput ByteString) -> Tokens (NoShareInput ByteString) -> [Token (NoShareInput ByteString)] # chunkLength :: Proxy (NoShareInput ByteString) -> Tokens (NoShareInput ByteString) -> Int # chunkEmpty :: Proxy (NoShareInput ByteString) -> Tokens (NoShareInput ByteString) -> Bool # take1_ :: NoShareInput ByteString -> Maybe (Token (NoShareInput ByteString), NoShareInput ByteString) # takeN_ :: Int -> NoShareInput ByteString -> Maybe (Tokens (NoShareInput ByteString), NoShareInput ByteString) # takeWhile_ :: (Token (NoShareInput ByteString) -> Bool) -> NoShareInput ByteString -> (Tokens (NoShareInput ByteString), NoShareInput ByteString) #
Stream (NoShareInput ByteString) # |
Instance details Defined in Text.Megaparsec.Stream Associated Types
type Token (NoShareInput ByteString) |
Instance details Defined in Text.Megaparsec.Stream type Token (NoShareInput ByteString) = Word8
type Tokens (NoShareInput ByteString) |
Instance details Defined in Text.Megaparsec.Stream type Tokens (NoShareInput ByteString) = ByteString
Methods tokenToChunk :: Proxy (NoShareInput ByteString) -> Token (NoShareInput ByteString) -> Tokens (NoShareInput ByteString) # tokensToChunk :: Proxy (NoShareInput ByteString) -> [Token (NoShareInput ByteString)] -> Tokens (NoShareInput ByteString) # chunkToTokens :: Proxy (NoShareInput ByteString) -> Tokens (NoShareInput ByteString) -> [Token (NoShareInput ByteString)] # chunkLength :: Proxy (NoShareInput ByteString) -> Tokens (NoShareInput ByteString) -> Int # chunkEmpty :: Proxy (NoShareInput ByteString) -> Tokens (NoShareInput ByteString) -> Bool # take1_ :: NoShareInput ByteString -> Maybe (Token (NoShareInput ByteString), NoShareInput ByteString) # takeN_ :: Int -> NoShareInput ByteString -> Maybe (Tokens (NoShareInput ByteString), NoShareInput ByteString) # takeWhile_ :: (Token (NoShareInput ByteString) -> Bool) -> NoShareInput ByteString -> (Tokens (NoShareInput ByteString), NoShareInput ByteString) #
Stream (NoShareInput Text) # |
Instance details Defined in Text.Megaparsec.Stream Associated Types
type Token (NoShareInput Text) |
Instance details Defined in Text.Megaparsec.Stream type Token (NoShareInput Text) = Char
type Tokens (NoShareInput Text) |
Instance details Defined in Text.Megaparsec.Stream type Tokens (NoShareInput Text) = Text
Methods tokenToChunk :: Proxy (NoShareInput Text) -> Token (NoShareInput Text) -> Tokens (NoShareInput Text) # tokensToChunk :: Proxy (NoShareInput Text) -> [Token (NoShareInput Text)] -> Tokens (NoShareInput Text) # chunkToTokens :: Proxy (NoShareInput Text) -> Tokens (NoShareInput Text) -> [Token (NoShareInput Text)] # chunkLength :: Proxy (NoShareInput Text) -> Tokens (NoShareInput Text) -> Int # chunkEmpty :: Proxy (NoShareInput Text) -> Tokens (NoShareInput Text) -> Bool # take1_ :: NoShareInput Text -> Maybe (Token (NoShareInput Text), NoShareInput Text) # takeN_ :: Int -> NoShareInput Text -> Maybe (Tokens (NoShareInput Text), NoShareInput Text) # takeWhile_ :: (Token (NoShareInput Text) -> Bool) -> NoShareInput Text -> (Tokens (NoShareInput Text), NoShareInput Text) #
Stream (NoShareInput Text) # |
Instance details Defined in Text.Megaparsec.Stream Associated Types
type Token (NoShareInput Text) |
Instance details Defined in Text.Megaparsec.Stream type Token (NoShareInput Text) = Char
type Tokens (NoShareInput Text) |
Instance details Defined in Text.Megaparsec.Stream type Tokens (NoShareInput Text) = Text
Methods tokenToChunk :: Proxy (NoShareInput Text) -> Token (NoShareInput Text) -> Tokens (NoShareInput Text) # tokensToChunk :: Proxy (NoShareInput Text) -> [Token (NoShareInput Text)] -> Tokens (NoShareInput Text) # chunkToTokens :: Proxy (NoShareInput Text) -> Tokens (NoShareInput Text) -> [Token (NoShareInput Text)] # chunkLength :: Proxy (NoShareInput Text) -> Tokens (NoShareInput Text) -> Int # chunkEmpty :: Proxy (NoShareInput Text) -> Tokens (NoShareInput Text) -> Bool # take1_ :: NoShareInput Text -> Maybe (Token (NoShareInput Text), NoShareInput Text) # takeN_ :: Int -> NoShareInput Text -> Maybe (Tokens (NoShareInput Text), NoShareInput Text) # takeWhile_ :: (Token (NoShareInput Text) -> Bool) -> NoShareInput Text -> (Tokens (NoShareInput Text), NoShareInput Text) #
type Token (NoShareInput ByteString) # |
Instance details Defined in Text.Megaparsec.Stream type Token (NoShareInput ByteString) = Word8
type Token (NoShareInput ByteString) # |
Instance details Defined in Text.Megaparsec.Stream type Token (NoShareInput ByteString) = Word8
type Token (NoShareInput Text) # |
Instance details Defined in Text.Megaparsec.Stream type Token (NoShareInput Text) = Char
type Token (NoShareInput Text) # |
Instance details Defined in Text.Megaparsec.Stream type Token (NoShareInput Text) = Char
type Tokens (NoShareInput ByteString) # |
Instance details Defined in Text.Megaparsec.Stream type Tokens (NoShareInput ByteString) = ByteString
type Tokens (NoShareInput ByteString) # |
Instance details Defined in Text.Megaparsec.Stream type Tokens (NoShareInput ByteString) = ByteString
type Tokens (NoShareInput Text) # |
Instance details Defined in Text.Megaparsec.Stream type Tokens (NoShareInput Text) = Text
type Tokens (NoShareInput Text) # |
Instance details Defined in Text.Megaparsec.Stream type Tokens (NoShareInput Text) = Text

class Stream s => VisualStream s where #

Type class for inputs that can also be used for debugging.

Since: 9.0.0

Minimal complete definition

showTokens

Methods

showTokens :: Proxy s -> NonEmpty (Token s) -> String #

Pretty-print non-empty stream of tokens. This function is also used to print single tokens (represented as singleton lists).

Since: 7.0.0

tokensLength :: Proxy s -> NonEmpty (Token s) -> Int #

Return the number of characters that a non-empty stream of tokens spans. The default implementation is sufficient if every token spans exactly 1 character.

Since: 8.0.0

#### Instances

Instances details

VisualStream ByteString # |
Instance details Defined in Text.Megaparsec.Stream Methods showTokens :: Proxy ByteString -> NonEmpty (Token ByteString) -> String # tokensLength :: Proxy ByteString -> NonEmpty (Token ByteString) -> Int #
VisualStream ByteString # |
Instance details Defined in Text.Megaparsec.Stream Methods showTokens :: Proxy ByteString -> NonEmpty (Token ByteString) -> String # tokensLength :: Proxy ByteString -> NonEmpty (Token ByteString) -> Int #
VisualStream Text # |
Instance details Defined in Text.Megaparsec.Stream Methods showTokens :: Proxy Text -> NonEmpty (Token Text) -> String # tokensLength :: Proxy Text -> NonEmpty (Token Text) -> Int #
VisualStream Text # |
Instance details Defined in Text.Megaparsec.Stream Methods showTokens :: Proxy Text -> NonEmpty (Token Text) -> String # tokensLength :: Proxy Text -> NonEmpty (Token Text) -> Int #
VisualStream String # |
Instance details Defined in Text.Megaparsec.Stream Methods showTokens :: Proxy String -> NonEmpty (Token String) -> String # tokensLength :: Proxy String -> NonEmpty (Token String) -> Int #

class Stream s => TraversableStream s where #

Type class for inputs that can also be used for error reporting.

Since: 9.0.0

Minimal complete definition

reachOffset | reachOffsetNoLine

Methods

reachOffset #

Arguments

:: Int | Offset to reach
-> PosState s | Initial `PosState` to use
-> (Maybe String, PosState s) | See the description of the function

Given an offset `o` and initial `PosState`, adjust the state in such a way that it starts at the offset.

Return two values (in order):

- `Maybe` `String` representing the line on which the given offset `o` is located. It can be omitted (i.e. `Nothing`); in that case error reporting functions will not show offending lines. If returned, the line should satisfy a number of conditions that are described below.
- The updated `PosState` which can be in turn used to locate another offset `o'` given that `o' >= o`.

The `String` representing the offending line in input stream should satisfy the following:

- It should adequately represent location of token at the offset of interest, that is, character at `sourceColumn` of the returned `SourcePos` should correspond to the token at the offset `o`.
- It should not include the newline at the end.
- It should not be empty, if the line happens to be empty, it should be replaced with the string `"<empty line>"`.
- Tab characters should be replaced by appropriate number of spaces, which is determined by the `pstateTabWidth` field of `PosState`.

Note: type signature of the function was changed in the version 9.0.0.

Since: 7.0.0

reachOffsetNoLine #

Arguments

:: Int | Offset to reach
-> PosState s | Initial `PosState` to use
-> PosState s | Reached source position and updated state

A version of `reachOffset` that may be faster because it doesn't need to fetch the line at which the given offset in located.

The default implementation is this:

    reachOffsetNoLine o pst =
      snd (reachOffset o pst)

Note: type signature of the function was changed in the version 8.0.0.

Since: 7.0.0

#### Instances

Instances details

TraversableStream ByteString # |
Instance details Defined in Text.Megaparsec.Stream Methods reachOffset :: Int -> PosState ByteString -> (Maybe String, PosState ByteString) # reachOffsetNoLine :: Int -> PosState ByteString -> PosState ByteString #
TraversableStream ByteString # |
Instance details Defined in Text.Megaparsec.Stream Methods reachOffset :: Int -> PosState ByteString -> (Maybe String, PosState ByteString) # reachOffsetNoLine :: Int -> PosState ByteString -> PosState ByteString #
TraversableStream Text # |
Instance details Defined in Text.Megaparsec.Stream Methods reachOffset :: Int -> PosState Text -> (Maybe String, PosState Text) # reachOffsetNoLine :: Int -> PosState Text -> PosState Text #
TraversableStream Text # |
Instance details Defined in Text.Megaparsec.Stream Methods reachOffset :: Int -> PosState Text -> (Maybe String, PosState Text) # reachOffsetNoLine :: Int -> PosState Text -> PosState Text #
TraversableStream String # |
Instance details Defined in Text.Megaparsec.Stream Methods reachOffset :: Int -> PosState String -> (Maybe String, PosState String) # reachOffsetNoLine :: Int -> PosState String -> PosState String #