PySmile: Compatibiltiy with Pylint?

[maghnie]

As part of my framework's Git CI pipeline, there's a code quality stage that uses Pylint.
However, the CI fails with these warnings and error:

Code: Select all

Problem importing module async.py: No module named '_pysmile'
Problem importing module bad_chained_comparison.py: No module named '_pysmile'
Problem importing module base: No module named '_pysmile'
Problem importing module base_checker.py: No module named '_pysmile'
Problem importing module classes: No module named '_pysmile'
Problem importing module dataclass_checker.py: No module named '_pysmile'
Problem importing module deprecated.py: No module named '_pysmile'
Problem importing module design_analysis.py: No module named '_pysmile'
Problem importing module dunder_methods.py: No module named '_pysmile'
Problem importing module ellipsis_checker.py: No module named '_pysmile'
Problem importing module exceptions.py: No module named '_pysmile'
Problem importing module format.py: No module named '_pysmile'
Problem importing module imports.py: No module named '_pysmile'
Problem importing module lambda_expressions.py: No module named '_pysmile'
Problem importing module logging.py: No module named '_pysmile'
Problem importing module method_args.py: No module named '_pysmile'
Problem importing module misc.py: No module named '_pysmile'
Problem importing module modified_iterating_checker.py: No module named '_pysmile'
Problem importing module nested_min_max.py: No module named '_pysmile'
Problem importing module newstyle.py: No module named '_pysmile'
Problem importing module non_ascii_names.py: No module named '_pysmile'
Problem importing module raw_metrics.py: No module named '_pysmile'
Problem importing module refactoring: No module named '_pysmile'
Problem importing module similar.py: No module named '_pysmile'
Problem importing module spelling.py: No module named '_pysmile'
Problem importing module stdlib.py: No module named '_pysmile'
Problem importing module strings.py: No module named '_pysmile'
Problem importing module threading_checker.py: No module named '_pysmile'
Problem importing module typecheck.py: No module named '_pysmile'
Problem importing module unicode.py: No module named '_pysmile'
Problem importing module unsupported_version.py: No module named '_pysmile'
Problem importing module utils.py: No module named '_pysmile'
Problem importing module variables.py: No module named '_pysmile'
Problem importing module base_reporter.py: No module named '_pysmile'
Problem importing module collecting_reporter.py: No module named '_pysmile'
Problem importing module json_reporter.py: No module named '_pysmile'
Problem importing module multi_reporter.py: No module named '_pysmile'
Problem importing module reports_handler_mix_in.py: No module named '_pysmile'
Problem importing module text.py: No module named '_pysmile'
Problem importing module ureports: No module named '_pysmile'
Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "d:\Repos\00-Phd\bayesian-tsc\.conda\Scripts\pylint.exe\__main__.py", line 8, in <module>
  File "d:\Repos\00-Phd\bayesian-tsc\.conda\Lib\site-packages\pylint\__init__.py", line 34, in run_pylint
    PylintRun(argv or sys.argv[1:])
  File "d:\Repos\00-Phd\bayesian-tsc\.conda\Lib\site-packages\pylint\lint\run.py", line 162, in __init__
    args = _config_initialization(
           ^^^^^^^^^^^^^^^^^^^^^^^
  File "d:\Repos\00-Phd\bayesian-tsc\.conda\Lib\site-packages\pylint\config\config_initialization.py", line 114, in _config_initialization
    for exc_name in linter.config.overgeneral_exceptions:
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AttributeError: 'Namespace' object has no attribute 'overgeneral_exceptions'

Is there a specific flag or parameter that can be passed on to Pylint, to let it handle PySmile properly?

I've tried ignoring import warnings but that didn't work. Currently, we just set up a different virtual environment that doesn't include PySmile to make the code linter work.

This workaround is acceptable. So it would just be a bonus if PySmile would work with Pylint too.

[piotr [BayesFusion]]

The problem found is not caused directly by Pylint - this one is compatible with the code in which there are PySMILE objects.
A series of errors - No module named '_pysmile' suggests that our library is imported as an alias - can you share the code and write more about the CI tools you use to deploy the project?

[maghnie]

Thanks for the prognosis!

Since I may not share the actual code, I've tried to recreate the error locally with just 2 files: tutorial1.py and an __init__.py
(

pylint_bayesfusion_mwe.zip

(1.33 KiB) Downloaded 161 times

).

To recreate the error:

1. Create and activate a conda virtual environment with Python 3.12.4
2. Run:

   Code: Select all

   pip install pylint && pylint .

3. Commands runs successfully with this output:

   Code: Select all

   ************* Module pylint_bayesfusion_mwe.tutorial1
   tutorial1.py:1:0: C0114: Missing module docstring (missing-module-docstring)
   tutorial1.py:1:0: E0401: Unable to import 'pysmile' (import-error)
   tutorial1.py:2:0: C0414: Import alias does not rename original package (useless-import-alias)
   tutorial1.py:2:0: E0401: Unable to import 'pysmile_license' (import-error)
   tutorial1.py:10:0: C0115: Missing class docstring (missing-class-docstring)
   tutorial1.py:32:8: C0103: Variable name "economyDef" doesn't conform to snake_case naming style (invalid-name)
   tutorial1.py:39:8: C0103: Variable name "successDef" doesn't conform to snake_case naming style (invalid-name)
   tutorial1.py:49:8: C0103: Variable name "forecastDef" doesn't conform to snake_case naming style (invalid-name)
   tutorial1.py:75:4: C0116: Missing function or method docstring (missing-function-docstring)
   tutorial1.py:75:4: R0913: Too many arguments (7/5) (too-many-arguments)
   tutorial1.py:75:35: W0622: Redefining built-in 'id' (redefined-builtin)
   tutorial1.py:10:0: R0903: Too few public methods (1/2) (too-few-public-methods)
   tutorial1.py:2:0: W0611: Unused pysmile_license imported as pysmile_license (unused-import)
   
   -----------------------------------
   Your code has been rated at 3.23/10

4. In the same terminal, install pysmile and rerun pylint

   Code: Select all

   pip install --index-url https://support.bayesfusion.com/pysmile-A/ pysmile && pylint .
   

5. Now the pylint command results in an error:

   Code: Select all

   Problem importing module async.py: No module named '_pysmile'
   Problem importing module bad_chained_comparison.py: No module named '_pysmile'
   Problem importing module base: No module named '_pysmile'
   Problem importing module base_checker.py: No module named '_pysmile'
   Problem importing module classes: No module named '_pysmile'
   Problem importing module dataclass_checker.py: No module named '_pysmile'
   Problem importing module deprecated.py: No module named '_pysmile'
   Problem importing module design_analysis.py: No module named '_pysmile'
   Problem importing module dunder_methods.py: No module named '_pysmile'
   Problem importing module ellipsis_checker.py: No module named '_pysmile'
   Problem importing module exceptions.py: No module named '_pysmile'
   Problem importing module format.py: No module named '_pysmile'
   Problem importing module imports.py: No module named '_pysmile'
   Problem importing module lambda_expressions.py: No module named '_pysmile'
   Problem importing module logging.py: No module named '_pysmile'
   Problem importing module method_args.py: No module named '_pysmile'
   Problem importing module misc.py: No module named '_pysmile'
   Problem importing module modified_iterating_checker.py: No module named '_pysmile'
   Problem importing module nested_min_max.py: No module named '_pysmile'
   Problem importing module newstyle.py: No module named '_pysmile'
   Problem importing module non_ascii_names.py: No module named '_pysmile'
   Problem importing module raw_metrics.py: No module named '_pysmile'
   Problem importing module refactoring: No module named '_pysmile'
   Problem importing module similar.py: No module named '_pysmile'
   Problem importing module spelling.py: No module named '_pysmile'
   Problem importing module stdlib.py: No module named '_pysmile'
   Problem importing module strings.py: No module named '_pysmile'
   Problem importing module threading_checker.py: No module named '_pysmile'
   Problem importing module typecheck.py: No module named '_pysmile'
   Problem importing module unicode.py: No module named '_pysmile'
   Problem importing module unsupported_version.py: No module named '_pysmile'
   Problem importing module utils.py: No module named '_pysmile'
   Problem importing module variables.py: No module named '_pysmile'
   Problem importing module base_reporter.py: No module named '_pysmile'
   Problem importing module collecting_reporter.py: No module named '_pysmile'
   Problem importing module json_reporter.py: No module named '_pysmile'
   Problem importing module multi_reporter.py: No module named '_pysmile'
   Problem importing module reports_handler_mix_in.py: No module named '_pysmile'
   Problem importing module text.py: No module named '_pysmile'
   Problem importing module ureports: No module named '_pysmile'
   Traceback (most recent call last):
     File "<frozen runpy>", line 198, in _run_module_as_main
     File "<frozen runpy>", line 88, in _run_code
     File "c:\Users\mmg\Downloads\pylint_bayesfusion_mwe\.conda\Scripts\pylint.exe\__main__.py", line 7, in <module>
     File "c:\Users\mmg\Downloads\pylint_bayesfusion_mwe\.conda\Lib\site-packages\pylint\__init__.py", line 34, in run_pylint
       PylintRun(argv or sys.argv[1:])
     File "c:\Users\mmg\Downloads\pylint_bayesfusion_mwe\.conda\Lib\site-packages\pylint\lint\run.py", line 162, in __init__
       args = _config_initialization(
              ^^^^^^^^^^^^^^^^^^^^^^^
     File "c:\Users\mmg\Downloads\pylint_bayesfusion_mwe\.conda\Lib\site-packages\pylint\config\config_initialization.py", line 114, in _config_initialization
       for exc_name in linter.config.overgeneral_exceptions:
                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
   AttributeError: 'Namespace' object has no attribute 'overgeneral_exceptions'
   

My local environment, as you can tell, is Windows. But the CI is running on Linux. So the issue is probably OS-independent.
Also, including or excluding the pysmile license file doesn't change the outcomes.

Any hints are appreciated!

[piotr [BayesFusion]]

It turned out that the fault lies with PySMILE installed by pip. When installing wheel in the modules directory (site-packages), PySMILE leaves an unnecessary __init__.py file. It has the same modification date as pysmile.pyd on Windows and pysmile.so on Linux.

It is safe to remove it manually, however, we will soon update our repository so that the unnecessary would not be part of wheel anymore.

After removing this file, any messages about an unknown module disappear and pylint works properly. Interestingly, outside the conda environment, everything works properly and the redundant __init__.py file does not affect anything.

[maghnie]

Will keep an eye out for the package update! Thank you for the explanation :)