In Loggly, when you send log data with custom field names (context) along with the log message, Loggly indexes these field names to make them searchable and filterable. However, there is a limit to the number of unique field names that Loggly indexes by default.

When you exceed the default unique field names limit (which is 1,000 unique field names), Loggly will stop indexing any additional unique field names. As a result, those field names will not be searchable or filterable in Loggly.

To overcome the indexing unique field names limit in Loggly, you have a few options:

  1. Refactor Context Format: One approach is to refactor your log data context format to use a smaller set of standard field names instead of many unique ones. For example, you can use a few key field names such as "user_id," "request_id," "timestamp," etc., and then attach their respective values in each log message.

    This refactoring will help you stay within the indexing limit while still providing the necessary context for your log messages. It might require changes to your logging implementation to consistently use the standardized field names across your logs.

  2. Loggly Dynamic Field Explorer: If you want to keep the flexibility of logging custom field names but are concerned about the indexing limit, you can use the "Dynamic Field Explorer" feature in Loggly. This feature allows you to discover and search the available fields without indexing all unique field names.

    You can find the "Dynamic Field Explorer" in the "Logs" section of Loggly's web interface. It enables you to explore and search fields even if they are not fully indexed.

  3. Field Aliases: Another option is to use field aliases to group different unique field names under a common alias. This way, you can apply filters and searches on the alias to access multiple unique field names.

    For example, you could have multiple field names like "user_1," "user_2," "user_3," etc., and then create a field alias called "user" that includes all these field names. This will allow you to perform searches and filters on the "user" alias, effectively searching across all unique "user" fields.

Keep in mind that each of these options has its trade-offs, and the choice depends on your specific use case and requirements. Refactoring the context format is likely the most straightforward and recommended solution to stay within Loggly's indexing limits and improve the log data's consistency and searchability.

Have questions or queries?
Get in Touch