refactor/query-workers (#528)

* add files that we'll need for this refactor

* add MerlinBaseFactory class

* refactor MerlinBackendFactory to use MerlinBaseFactory

* add tests for MerlinBaseFactory and fix backend tests

* convert monitor factory to use new MerlinBaseFactory

* remove comment and update MonitorFactory docstring

* update MerlinStatusRendererFactory to use MerlinBaseFactory and fix TODOs related to Maestro

* add tests for the status renderer factory

* update CHANGELOG

* run fix-style

* fix issue with typehint that breaks in python 3.8

* mocked more items to try to fix broken tests on github

* create MerlinWorker and CeleryWorker classes to handle the launching of workers

* implement worker-handler related classes

* add worker factory class and small cleanup to the rest of the worker files

* remove functions that are now in the new worker files

* link the new worker classes to the actual launching of workers

* fix regex in test

* remove watchdog files and run fix-style

* fix tests that broke after refactor

* run fix-style

* update CHANGELOG

* refactored how the database command is initially processed

* add filtering logic to the entity managers

* add support for database-level filtering

* move get_plural_of_entity and get_singular_of_entity to utils.py

* fix database command tests

* fix broken tests for entity manager

* run fix-style and update CHANGELOG

* add tests for new backend filtering

* add tests for new cli utils functions

* fix typo in cli/utils.py

* run fix-style

* add filter options to the command line page

* resolve conflicts

* resolve conflicts

* mocked more items to try to fix broken tests on github

* create MerlinWorker and CeleryWorker classes to handle the launching of workers

* implement worker-handler related classes

* add worker factory class and small cleanup to the rest of the worker files

* remove functions that are now in the new worker files

* link the new worker classes to the actual launching of workers

* add files that we'll need for this refactor

* fix regex in test

* remove watchdog files and run fix-style

* fix tests that broke after refactor

* run fix-style

* update CHANGELOG

* run fix-style

* fix worker-related factory classes

* add tests for new workers files

* change imports for Celery worker and handler in tests

* run fix-style

* attempt to fix broken unit tests

* fix style

* add mocked merlin db to broken test

* run fix-style

* fix susbcriptable error in test

* first pass at adding new worker formatter classes

* link new formatters to new query-workers refactor

* remove _discover_builtins method that's not used and annoying

* change database to only store base name of queue

* get filters working for query-workers command

* finalize worker formatters

* add console as class attribute for formmatters and fix pylint issues

* more pylint fixes

* add attributes section to json formatter

* fix broken tests

* update CHANGELOG

* updated the query-workers docs page

* update the query-workers CLI docs

* fix one more merge problem

* add query-workers tests to celery worker handler test file

* add tests for rich formatter

* add tests for all formatter modules

* run fix-style

* fix broken test

* remove query-workers integration tests

* fix issue with tests

* run fix style

* fix issue with pid and use WorkerStatus enum properly

* add functionality to compare db data against live celery data

* run fix style and fix tests

* move function to CeleryWorkerHandler and add/fix tests

* run fix-style

* integrate new queue-info code with latest changes

* add ability to query workers from local db

* update docs to add new local-db option

* remove accidental double reload_data call

* fix bug when loading status of a worker

* fix two unit tests related to latest changes

Author: bgunnar5

CHANGELOG.md

@@ -14,6 +14,18 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
   - Can be disabled with `--disable-gc`
 - Alias for `merlin database` command so it can be called with `merlin db`
 - Status of run entities in the database (this will differ from task statuses)
+- New classes for formatting `query-workers` output:
+  - `WorkerFormatter`: base class for defining formatted output for workers
+  - `RichWorkerFormatter`: implementation of `WorkerFormatter` for output using the rich library
+  - `JSONWorkerFormatter`: implementation of `WorkerFormatter` for JSON output
+  - `WorkerFormatterFactory`: factory class for selecting the desired worker formatter
+
+### Changed
+- Changes to the `query-workers` command:
+  - Output now displays a lot more information, including logical and physical worker specific info
+  - Output now formatted using rich tables or json
+  - Now handled through worker classes rather than functions in `celeryadapter.py` file
+  - Behind the scenes this is now querying the new Merlin Database
 
 ## [2.0.0b2]