i1UdZgdZddlZddlZddlmZddlZddlZddl Z ddl m Z m Z ejr ddlmZddlmZdd lmZdd lmZdd lmZmZe j2e d d dZe ed < e j8Zej=dZej=dde fdZ!GddZ"GddejFZ$y)aImplementations of the framework for the SearXNG engines. - :py:obj:`searx.enginelib.EngineCache` - :py:obj:`searx.enginelib.Engine` - :py:obj:`searx.enginelib.traits` There is a command line for developer purposes and for deeper analysis. Here is an example in which the command line is called in the development environment:: $ ./manage pyenv.cmd bash --norc --noprofile (py3) python -m searx.enginelib --help .. hint:: The long term goal is to modularize all implementations of the engine framework here in this Python package. ToDo: - move implementations of the :ref:`searx.engines loader` to a new module in the :py:obj:`searx.enginelib` namespace. ----- ) EngineCacheEngine ENGINES_CACHEN)Callable)ExpireCacheSQLiteExpireCacheCfg)traits) EngineTraits) SXNG_Response) EngineResults)OfflineParamTypesOnlineParamTypesri: i)name MAXHOLD_TIMEMAINTENANCE_PERIODcxd}t|tdt|zttjj tdtj j }t|tdt|ztttjy)z)Show state for the caches of the engines.zcache tables and key/values=zproperties of N) printlenrstatereportcfgrstr properties)titles )/root/searxng/searx/enginelib/__init__.pyrr:s *E %L #E  -    & & () G]..334 5E %L #E  #m&& '(forcec0tj|y)z.Carry out maintenance on cache of the engines.rN)r maintenancer!s rr"r"IsE*rc eZdZdZddededzfdZddedejdedzde fd Z dded ejdejfd Z d ee zdefd Z y)raePersistent (SQLite) key/value cache that deletes its values again after ``expire`` seconds (default/max: :py:obj:`MAXHOLD_TIME `). This class is a wrapper around :py:obj:`ENGINES_CACHE` (:py:obj:`ExpireCacheSQLite `). In the :origin:`searx/engines/demo_offline.py` engine you can find an exemplary implementation of such a cache other examples are implemented in: - :origin:`searx/engines/radio_browser.py` - :origin:`searx/engines/soundcloud.py` - :origin:`searx/engines/startpage.py` .. code: python from searx.enginelib import EngineCache CACHE: EngineCache def init(engine_settings): global CACHE CACHE = EngineCache(engine_settings["name"]) def request(query, params): token = CACHE.get(key="token") if token is None: token = get_token() # cache token of this engine for 1h CACHE.set(key="token", value=token, expire=3600) ... For introspection of the DB, jump into developer environment and run command to show cache state:: $ ./manage pyenv.cmd bash --norc --noprofile (py3) python -m searx.enginelib cache state cache tables and key/values =========================== [demo_offline ] 2025-04-22 11:32:50 count --> (int) 4 [startpage ] 2025-04-22 12:32:30 SC_CODE --> (str) fSOBnhEMlDfE20 [duckduckgo ] 2025-04-22 12:32:31 4dff493e.... --> (str) 4-128634958369380006627592672385352473325 [duckduckgo ] 2025-04-22 12:40:06 3e2583e2.... --> (str) 4-263126175288871260472289814259666848451 [radio_browser ] 2025-04-23 11:33:08 servers --> (list) ['https://de2.api.radio-browser.info', ...] [soundcloud ] 2025-04-29 11:40:06 guest_client_id --> (str) EjkRJG0BLNEZquRiPZYdNtJdyGtTuHdp [wolframalpha ] 2025-04-22 12:40:06 code --> (str) 5aa79f86205ad26188e0e26e28fb7ae7 number of tables: 6 number of key/value pairs: 7 In the "cache tables and key/values" section, the table name (engine name) is at first position on the second there is the calculated expire date and on the third and fourth position the key/value is shown. About duckduckgo: The *vqd coode* of ddg depends on the query term and therefore the key is a hash value of the query term (to not to store the raw query term). In the "properties of ENGINES_CACHE" section all properties of the SQLiteAppl / ExpireCache and their last modification date are shown:: properties of ENGINES_CACHE =========================== [last modified: 2025-04-22 11:32:27] DB_SCHEMA : 1 [last modified: 2025-04-22 11:32:27] LAST_MAINTENANCE : [last modified: 2025-04-22 11:32:27] crypt_hash : ca612e3566fdfd7cf7efe... [last modified: 2025-04-22 11:32:30] CACHE-TABLE--demo_offline: demo_offline [last modified: 2025-04-22 11:32:30] CACHE-TABLE--startpage: startpage [last modified: 2025-04-22 11:32:31] CACHE-TABLE--duckduckgo: duckduckgo [last modified: 2025-04-22 11:33:08] CACHE-TABLE--radio_browser: radio_browser [last modified: 2025-04-22 11:40:06] CACHE-TABLE--soundcloud: soundcloud [last modified: 2025-04-22 11:40:06] CACHE-TABLE--wolframalpha: wolframalpha These properties provide information about the state of the ExpireCache and control the behavior. For example, the maintenance intervals are controlled by the last modification date of the LAST_MAINTENANCE property and the hash value of the password can be used to detect whether the password has been changed (in this case the DB entries can no longer be decrypted and the entire cache must be discarded). N engine_nameexpirec|xstjj|_dtj ztj z}dj|Dcgc] }||vr|nd c}|_ycc}w)Nz-_._) rrrr%string ascii_lettersdigitsjoin table_name)selfr$r%_validcs r__init__zEngineCache.__init__s\!C]%6%6%C%C --- =!ww;'WaQ&[c(A'WX'WsA3keyvaluereturncdtj|||xs |j|jS)N)r2r3r%ctx)rsetr%r-)r.r2r3r%s rr7zEngineCache.sets3  (T[[ !  rdefaultcFtj|||jS)N)r8r6)rgetr-)r.r2r8s rr:zEngineCache.gets  g4?? KKrrc.tj|S)N)r)r secret_hash)r.rs rr<zEngineCache.secret_hashs((d(33r)N)__name__ __module__ __qualname____doc__rintr1tAnyboolr7r:bytesr<rrrrOsM^YCYtY  s 155 #*  LsLQUULaeeL4e 44rrceZdZUdZej ed<eed< eed< dZ e ed< eed< eed< eed < eed < eed < d ed < ded< e eed< eed< eed< eed< eed< e ed< eed< e ee eeffed< eed< eed< e ee eeffed< eed< eed< e eed< e ed< de eejfd efd!Zde eejfd ed"zfd#Zej&d$ed%d&d d'fd(Zej&d$ed%d)d d"fd*Zej&d,d+Zy")-rzClass of engine instances build from YAML settings. Further documentation see :ref:`general engine configuration`. .. hint:: This class is currently never initialized and only used for type hinting. logger engine_typepagingrmax_pagetime_range_support safesearchlanguage_supportlanguageregionz$Callable[[EngineTraits, bool], None] fetch_traitsztraits.EngineTraitsr categoriesrengine enable_httpshortcuttimeoutdisplay_error_messagesproxiesdisabledinactiveaboutusing_tor_proxysend_accept_language_headertokensweightengine_settingsr4cy)aDynamic setup of the engine settings. With this method, the engine's setup is carried out. For example, to check or dynamically adapt the values handed over in the parameter ``engine_settings``. The return value (True/False) indicates whether the setup was successful and the engine can be built or rejected. The method is optional and is called synchronously as part of the initialization of the service and is therefore only suitable for simple (local) exams/changes at the engine setting. The :py:obj:`Engine.init` method must be used for longer tasks in which values of a remote must be determined, for example. TrFr.r`s rsetupz Engine.setup8srNcy)aYInitialization of the engine. The method is optional and asynchronous (in a thread). It is suitable, for example, for setting up a cache (for the engine) or for querying values (required by the engine) from a remote. Whether the initialization was successful can be indicated by the return value ``True`` or even ``False``. - If no return value is given from this init method (``None``), this is equivalent to ``True``. - If an exception is thrown as part of the initialization, this is equivalent to ``False``. TrFrbs rinitz Engine.initHs rqueryparamsrr cy)z(Search method of the ``offline`` enginesNrFr.rfrgs rsearchz Engine.searchZrrcy)zOMethod to build the parameters for the request of an ``online`` engine.NrFris rrequestzEngine.request^rkrcy)z5Method to parse the response of an ``online`` engine.NrF)r.resps rresponsezEngine.responsecrkr)ror r4r )r=r>r?r@loggingLogger__annotations__rrDrKrAlistfloatdictrBrCrcreabcabstractmethodrjrmrprFrrrrs NN= L)HcH,$5M K98; !!S Q I K9M7 N-  / #tCH~% &&N5NE T#s(^# $$ H!%%3 I KUT#quu*%5$ Daee$4$ 7C7)<777 S*< DDrr)T)%r@__all__typingrBrwcollections.abcrrqr)typercacherr TYPE_CHECKINGsearx.enginelibr searx.enginelib.traitsr searx.extended_typesr searx.result_typesr searx.search.processorsrr build_cacherrsTyperappcommandrrDr"rABCrrFrrrs0 5 $ 5??&320K#@#4#@#@ %"$  -ekkm ) )+t++ a4a4HrDSWWrDr