Answer

Background:

Lexical analysis is the first stage of Perl code interpretation. At this stage, the source text is broken down into "lexemes": variables, keywords, operators, literals, etc. One feature of Perl is that the syntax of the language is very flexible; many constructs allow for variability, and variables intermingle with operators and identifiers.

Problem:

The main challenge in Perl's lexical analysis is the ambiguity of parsing and the danger of substituting variable/string values directly into code, especially in constructs like eval and with variable interpolation in strings with files, paths, and regular expressions. Perl does not perform "double parsing": if you are forming a string for eval, the parsing starts from scratch, which can lead to unexpected errors and vulnerabilities (SQL injections for DBI, erroneous syntax, etc.).

Solution:

To avoid unexpected effects during lexical analysis, one should write the most explicit code possible, use strict formatting via use strict and warnings, avoid unnecessary eval, and express dynamics not through string interpolation, but structurally: through subroutines, closures, and strong data typing. For complex interpolations, it's often best to use sprintf, sprintf-like formats, and modules like Text::Template.

Code example:

my $operation = 'print';
my $argument  = 'Hello, world!';
# Never do this:
eval "$operation \"$argument\";";  # Dangerous!
# Better:
if ($operation eq 'print') {
    print $argument;
}

Key features:

The lexical analyzer automaton in Perl is complex and non-trivial, affecting bug hunting
Ambiguities may occur in Perl during substitutions, requiring an experienced approach
The use of strict modes and explicit expressions helps prevent errors

Trick Questions.

Trick Question 1: "Can we simply escape single and double quotes inside a string to avoid vulnerabilities when using eval?"

In fact, simply escaping quotes does not protect against all possible vulnerabilities, as the Perl parser interprets the string as a whole, and complex nested constructs may bypass escaping. Use a structural approach.

Trick Question 2: "Can here-doc be used for safely generating Perl code for eval?"

Here-doc makes formatting long strings easier, but does not guard against syntax errors and is still vulnerable to injections if unvalidated data is inserted. It does not provide security.

Trick Question 3: "Does Perl read expressions in strings that are not directly passed to eval?"

No, Perl parses only executable code; strings outside of eval or similar mechanisms are not parsed and not executed.

Common Mistakes and Anti-patterns

Directly substituting variables into eval
Inadequately checking content dynamically generated for execution
Absence of use strict and use warnings

Real-life Example

Negative Case

A programmer forms a dynamic SQL query through a string, substituting user parameters without validation, and subsequently tries to execute this query via eval.

Pros:

Works quickly in trivial cases

Cons:

Vulnerable to injection
Parsing errors due to unexpected characters

Positive Case

Instead of dynamic eval, prepared statements are used in DBI, parameters are substituted via placeholders, and errors are covered with use strict and warnings.