[arch][snap] improvement -> caching categories file for 24 hours

CHANGELOG.md

@@ -38,12 +38,22 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/).
         - the index is now stored at **~/.cache/bauh/arch/aur/index.txt**.
     - info window
         - date fields format changed to numbers (e.g: Thu Dec 17 17:19:55 2020 -> 2020-12-17 17:19:55)
+    - new settings property "categories_exp": it defines the expiration time (in HOURS) of the packages categories mapping file stored in disc. Default: 24 hours.
+    <p align="center">
+        <img src="https://raw.githubusercontent.com/vinifmor/bauh-files/master/pictures/releases/0.9.11/cats_exp.png">
+    </p>
 - Core
     - new settings property **boot.load_apps** (General -> Load apps after startup) that allows the user to choose if the installed packages/suggestions should be loaded on the management panel after the initialization process. Default: true.
     <p align="center">
         <img src="https://raw.githubusercontent.com/vinifmor/bauh-files/master/pictures/releases/0.9.11/load_apps.png">
     </p>
 
+- Snap
+    - new settings property "categories_exp": it defines the expiration time (in HOURS) of the Snaps categories mapping file stored in disc. Default: 24 hours.
+    <p align="center">
+        <img src="https://raw.githubusercontent.com/vinifmor/bauh-files/master/pictures/releases/0.9.11/cats_exp.png">
+    </p>
+
 - Web
     - now the environment settings are cached for 24 hours. This period can be controlled through the new settings property **environment.update_interval** (in minutes -> default: 1440 = 24 hours. Use **0** so that they are always updated).
     <p align="center">

README.md

@@ -128,6 +128,7 @@ installation_level: null # defines a default installation level: user or system.
 - The configuration file is located at **~/.config/bauh/snap.yml** and it allows the following customizations:
 ```
 install_channel: false  # it allows to select an available channel during the application installation. Default: false
+categories_exp: 24  # It defines the expiration time (in HOURS) of the Snaps categories mapping file stored in disc. Use 0 so that it is always updated during initialization.
 ```
 - Required dependencies:
     - Any distro: **snapd** ( it must be enabled after its installation. Details at https://snapcraft.io/docs/installing-snapd )
@@ -209,6 +210,7 @@ aur_idx_exp: 720  # It defines the period (in minutes) for the AUR index stored
 check_dependency_breakage: true # if, during the verification of the update requirements, specific versions of dependencies must also be checked. Example: package A depends on version 1.0 of B. If A and B were selected to upgrade, and B would be upgrade to 2.0, then B would be excluded from the transaction. Default: true.
 suggest_unneeded_uninstall: false  # if the dependencies apparently no longer necessary associated with the uninstalled packages should be suggested for uninstallation. When this property is enabled it automatically disables the property 'suggest_optdep_uninstall'. Default: false (to prevent new users from making mistakes)
 suggest_optdep_uninstall: false  # if the optional dependencies associated with uninstalled packages should be suggested for uninstallation. Only the optional dependencies that are not dependencies of other packages will be suggested. Default: false (to prevent new users from making mistakes)
+categories_exp: 24  # It defines the expiration time (in HOURS) of the packages categories mapping file stored in disc. Use 0 so that it is always updated during initialization.
 ```
 - Required dependencies:
     - **pacman**

bauh/commons/category.py

@@ -1,44 +1,64 @@
 import logging
 import os
+import time
 import traceback
+from datetime import datetime, timedelta
 from pathlib import Path
 from threading import Thread
-from typing import Dict, List
+from typing import Dict, List, Optional
 
 import requests
 
 from bauh.api.abstract.controller import SoftwareManager
 from bauh.api.http import HttpClient
+from bauh.commons.internet import InternetChecker
+from bauh.commons.util import map_timestamp_file
 
 
 class CategoriesDownloader(Thread):
 
     def __init__(self, id_: str, http_client: HttpClient, logger: logging.Logger, manager: SoftwareManager,
-                 url_categories_file: str, disk_cache_dir: str, categories_path: str, before=None, after=None):
+                 url_categories_file: str, categories_path: str, expiration: int, internet_checker: InternetChecker,
+                 internet_connection: Optional[bool] = True, before=None, after=None):
+        """
+        :param id_:
+        :param http_client:
+        :param logger:
+        :param manager:
+        :param url_categories_file:
+        :param categories_path:
+        :param expiration: cached file expiration in hours
+        :param internet_checker
+        :param before:
+        :param after:
+        """
         super(CategoriesDownloader, self).__init__(daemon=True)
         self.id_ = id_
         self.http_client = http_client
         self.logger = logger
         self.manager = manager
         self.url_categories_file = url_categories_file
-        self.disk_cache_dir = disk_cache_dir
         self.categories_path = categories_path
         self.before = before
         self.after = after
+        self.expiration = expiration
+        self.internet_connection = internet_connection
+        self.internet_checker = internet_checker
 
     def _msg(self, msg: str):
-        return '{}({}): {}'.format(self.__class__.__name__, self.id_, msg)
+        return '{} [{}]: {}'.format(self.__class__.__name__, self.id_, msg)
 
     def _read_categories_from_disk(self) -> Dict[str, List[str]]:
         if os.path.exists(self.categories_path):
-            self.logger.info(self._msg("Reading cached categories from the disk"))
+            self.logger.info(self._msg("Reading cached categories file {}".format(self.categories_path)))
 
             with open(self.categories_path) as f:
                 categories = f.read()
 
             return self._map_categories(categories)
-
+        else:
-        return {}
+            self.logger.warning("No cached categories file {} found".format(self.categories_path))
+            return {}
 
     def _map_categories(self, categories: str) -> Dict[str, List[str]]:
         categories_map = {}
@@ -49,16 +69,22 @@ class CategoriesDownloader(Thread):
 
         return categories_map
 
-    def _cache_categories_to_disk(self, categories: str):
+    def _cache_categories_to_disk(self, categories_str: str, timestamp: float):
-        self.logger.info(self._msg('Caching categories to the disk'))
+        self.logger.info(self._msg('Caching downloaded categories to disk'))
 
         try:
-            Path(self.disk_cache_dir).mkdir(parents=True, exist_ok=True)
+            Path(os.path.dirname(self.categories_path)).mkdir(parents=True, exist_ok=True)
 
             with open(self.categories_path, 'w+') as f:
-                f.write(categories)
+                f.write(categories_str)
 
-            self.logger.info(self._msg("Categories cached to the disk as '{}'".format(self.categories_path)))
+            self.logger.info(self._msg("Categories cached to file '{}'".format(self.categories_path)))
+
+            categories_ts_path = map_timestamp_file(self.categories_path)
+            with open(categories_ts_path, 'w+') as f:
+                f.write(str(timestamp))
+
+            self.logger.info(self._msg("Categories timestamp ({}) cached to file '{}'".format(timestamp, categories_ts_path)))
         except:
             self.logger.error(self._msg("Could not cache categories to the disk as '{}'".format(self.categories_path)))
             traceback.print_exc()
@@ -67,49 +93,82 @@ class CategoriesDownloader(Thread):
         self.logger.info(self._msg('Downloading category definitions from {}'.format(self.url_categories_file)))
 
         try:
+            timestamp = datetime.utcnow().timestamp()
             res = self.http_client.get(self.url_categories_file)
-
-            if res:
-                try:
-                    categories = self._map_categories(res.text)
-                    self.logger.info(self._msg('Loaded categories for {} applications'.format(len(categories))))
-
-                    if categories:
-                        Thread(target=self._cache_categories_to_disk, args=(res.text,), daemon=True).start()
-
-                    return categories
-                except:
-                    self.logger.error(self._msg("Could not parse categories definitions"))
-                    traceback.print_exc()
-            else:
-                self.logger.info(self._msg('Could not download {}'.format(self.url_categories_file)))
-
         except requests.exceptions.ConnectionError:
-            self.logger.warning(self._msg('The internet connection seems to be off.'))
+            self.logger.error(self._msg('[{}] Could not download categories. The internet connection seems to be off.'.format(self.id_)))
+            return {}
 
-        return {}
+        if not res:
+            self.logger.info(self._msg('Could not download {}'.format(self.url_categories_file)))
+            return {}
+
+        try:
+            categories = self._map_categories(res.text)
+            self.logger.info(self._msg('Loaded categories for {} applications'.format(len(categories))))
+        except:
+            self.logger.error(self._msg("Could not parse categories definitions"))
+            traceback.print_exc()
+            return {}
 
-    def _set_categories(self, categories: dict):
         if categories:
-            self.logger.info(self._msg("Settings {} categories to {}".format(len(categories), self.manager.__class__.__name__)))
+            self._cache_categories_to_disk(categories_str=res.text, timestamp=timestamp)
-            self.manager.categories = categories
 
-    def _download_and_set(self):
+        return categories
-        self._set_categories(self.download_categories())
+
+    def should_download(self) -> bool:
+        if self.internet_connection is False or (self.internet_connection is None and not self.internet_checker.is_available()):
+            self.logger.warning(self._msg("No internet connection. The categories file '{}' cannot be updated.".format(self.categories_path)))
+            return False
+
+        if self.expiration <= 0:
+            self.logger.warning(self._msg("No expiration set for the categories file '{}'. It should be downloaded".format(self.categories_path)))
+            return True
+
+        if not os.path.exists(self.categories_path):
+            self.logger.warning(self._msg("Categories file '{}' does not exist. It should be downloaded.".format(self.categories_path)))
+            return True
+
+        categories_ts_path = map_timestamp_file(self.categories_path)
+
+        if not os.path.exists(categories_ts_path):
+            self.logger.warning(self._msg("Categories timestamp file '{}' does not exist. The categories file should be re-downloaded.".format(categories_ts_path)))
+            return True
+
+        with open(categories_ts_path) as f:
+            timestamp_str = f.read()
+
+        try:
+            categories_timestamp = datetime.fromtimestamp(float(timestamp_str))
+        except:
+            self.logger.error(self._msg("An exception occurred when trying to parse the categories file timestamp from '{}'. The categories file should be re-downloaded.".format(categories_ts_path)))
+            traceback.print_exc()
+            return True
+
+        should_download = (categories_timestamp + timedelta(hours=self.expiration) <= datetime.utcnow())
+
+        if should_download:
+            self.logger.info(self._msg("Cached categories file '{}' has expired. A new one should be downloaded.".format(self.categories_path)))
+            return True
+        else:
+            self.logger.info(self._msg("Cached categories file '{}' is up to date. No need to re-download it.".format(self.categories_path)))
+            return False
 
     def run(self):
+        ti = time.time()
         if self.before:
             self.before()
 
-        cached = self._read_categories_from_disk()
+        should_download = self.should_download()
 
-        if cached:
+        if not should_download:
-            self._set_categories(cached)
+            cached = self._read_categories_from_disk()
-            Thread(target=self._download_and_set, daemon=True).start()
+            self.manager.categories = cached
         else:
-            self._download_and_set()
+            self.download_categories()
 
         if self.after:
             self.after()
 
-        self.logger.info(self._msg('Finished'))
+        tf = time.time()
+        self.logger.info(self._msg('Finished. Took {0:.2f} seconds'.format(tf - ti)))

bauh/commons/util.py

@@ -30,3 +30,8 @@ def size_to_byte(size: float, unit: str) -> int:
 
 def datetime_as_milis(date: datetime = datetime.utcnow()) -> int:
     return int(round(date.timestamp() * 1000))
+
+
+def map_timestamp_file(file_path: str) -> str:
+    path_split = file_path.split('/')
+    return '/'.join(path_split[0:-1]) + '/' + path_split[-1].split('.')[0] + '.ts'

bauh/gems/arch/config.py

@@ -22,7 +22,8 @@ def read_config(update_file: bool = False) -> dict:
                 'check_dependency_breakage': True,
                 'suggest_unneeded_uninstall': False,
                 'suggest_optdep_uninstall': False,
-                'aur_idx_exp': 720}
+                'aur_idx_exp': 720,
+                'categories_exp': 24}
     return read(CONFIG_FILE, template, update_file=update_file)
 
 

bauh/gems/arch/controller.py

@@ -2623,7 +2623,7 @@ class ArchManager(SoftwareManager):
         return action != SoftwareAction.SEARCH
 
     def _start_category_task(self, task_man: TaskManager):
-        task_man.register_task('arch_aur_cats', self.i18n['task.download_categories'].format('Arch'), get_icon_path())
+        task_man.register_task('arch_aur_cats', self.i18n['task.download_categories'], get_icon_path())
         task_man.update_progress('arch_aur_cats', 50, None)
 
     def _finish_category_task(self, task_man: TaskManager):
@@ -2654,8 +2654,11 @@ class ArchManager(SoftwareManager):
             self.disk_cache_updater.start()
 
         CategoriesDownloader(id_='Arch', http_client=self.context.http_client, logger=self.context.logger,
-                             manager=self, url_categories_file=URL_CATEGORIES_FILE, disk_cache_dir=ARCH_CACHE_PATH,
+                             manager=self, url_categories_file=URL_CATEGORIES_FILE,
                              categories_path=CATEGORIES_FILE_PATH,
+                             expiration=arch_config['categories_exp'] if isinstance(arch_config['categories_exp'], int) else '',
+                             internet_connection=internet_available,
+                             internet_checker=self.context.internet_checker,
                              before=lambda: self._start_category_task(task_manager),
                              after=lambda: self._finish_category_task(task_manager)).start()
 
@@ -2878,7 +2881,14 @@ class ArchManager(SoftwareManager):
                                  max_width=max_width,
                                  file_path=local_config['aur_build_dir'],
                                  capitalize_label=False,
-                                 directory=True)
+                                 directory=True),
+            TextInputComponent(id_='arch_cats_exp',
+                               label=self.i18n['arch.config.categories_exp'],
+                               tooltip=self.i18n['arch.config.categories_exp.tip'],
+                               max_width=max_width,
+                               only_int=True,
+                               capitalize_label=False,
+                               value=local_config['categories_exp'] if isinstance(local_config['categories_exp'], int) else ''),
         ]
 
         return PanelComponent([FormComponent(fields, spaces=False)])
@@ -2886,25 +2896,26 @@ class ArchManager(SoftwareManager):
     def save_settings(self, component: PanelComponent) -> Tuple[bool, Optional[List[str]]]:
         config = read_config()
 
-        form_install = component.components[0]
+        panel = component.components[0]
-        config['repositories'] = form_install.get_component('repos').get_selected()
+        config['repositories'] = panel.get_component('repos').get_selected()
-        config['aur'] = form_install.get_component('aur').get_selected()
+        config['aur'] = panel.get_component('aur').get_selected()
-        config['optimize'] = form_install.get_component('opts').get_selected()
+        config['optimize'] = panel.get_component('opts').get_selected()
-        config['sync_databases'] = form_install.get_component('sync_dbs').get_selected()
+        config['sync_databases'] = panel.get_component('sync_dbs').get_selected()
-        config['sync_databases_startup'] = form_install.get_component('sync_dbs_start').get_selected()
+        config['sync_databases_startup'] = panel.get_component('sync_dbs_start').get_selected()
-        config['clean_cached'] = form_install.get_component('clean_cached').get_selected()
+        config['clean_cached'] = panel.get_component('clean_cached').get_selected()
-        config['refresh_mirrors_startup'] = form_install.get_component('ref_mirs').get_selected()
+        config['refresh_mirrors_startup'] = panel.get_component('ref_mirs').get_selected()
-        config['mirrors_sort_limit'] = form_install.get_component('mirrors_sort_limit').get_int_value()
+        config['mirrors_sort_limit'] = panel.get_component('mirrors_sort_limit').get_int_value()
-        config['repositories_mthread_download'] = form_install.get_component('mthread_download').get_selected()
+        config['repositories_mthread_download'] = panel.get_component('mthread_download').get_selected()
-        config['automatch_providers'] = form_install.get_component('autoprovs').get_selected()
+        config['automatch_providers'] = panel.get_component('autoprovs').get_selected()
-        config['edit_aur_pkgbuild'] = form_install.get_component('edit_aur_pkgbuild').get_selected()
+        config['edit_aur_pkgbuild'] = panel.get_component('edit_aur_pkgbuild').get_selected()
-        config['aur_remove_build_dir'] = form_install.get_component('aur_remove_build_dir').get_selected()
+        config['aur_remove_build_dir'] = panel.get_component('aur_remove_build_dir').get_selected()
-        config['aur_build_dir'] = form_install.get_component('aur_build_dir').file_path
+        config['aur_build_dir'] = panel.get_component('aur_build_dir').file_path
-        config['aur_build_only_chosen'] = form_install.get_component('aur_build_only_chosen').get_selected()
+        config['aur_build_only_chosen'] = panel.get_component('aur_build_only_chosen').get_selected()
-        config['aur_idx_exp'] = form_install.get_component('aur_idx_exp').get_int_value()
+        config['aur_idx_exp'] = panel.get_component('aur_idx_exp').get_int_value()
-        config['check_dependency_breakage'] = form_install.get_component('check_dependency_breakage').get_selected()
+        config['check_dependency_breakage'] = panel.get_component('check_dependency_breakage').get_selected()
-        config['suggest_optdep_uninstall'] = form_install.get_component('suggest_optdep_uninstall').get_selected()
+        config['suggest_optdep_uninstall'] = panel.get_component('suggest_optdep_uninstall').get_selected()
-        config['suggest_unneeded_uninstall'] = form_install.get_component('suggest_unneeded_uninstall').get_selected()
+        config['suggest_unneeded_uninstall'] = panel.get_component('suggest_unneeded_uninstall').get_selected()
+        config['categories_exp'] = panel.get_component('arch_cats_exp').get_int_value()
 
         if not config['aur_build_dir']:
             config['aur_build_dir'] = None

bauh/gems/arch/resources/locale/ca

@@ -41,6 +41,8 @@ arch.config.aur_build_only_chosen=Build only chosen (AUR)
 arch.config.aur_build_only_chosen.tip=Some AUR packages have a common PKGBUILD shared with other packages and that defines build instructions for each one. This property enabled will ensure that only the chosen package will be built.
 arch.config.aur_remove_build_dir=Remove build directory (AUR)
 arch.config.aur_remove_build_dir.tip=If a package's generated build directory should be removed after the operations is finished.
+arch.config.categories_exp=Categories expiration
+arch.config.categories_exp.tip=It defines the expiration time (in HOURS) of the packages categories mapping file stored in disc. Use 0 so that it is always updated during initialization.
 arch.config.clean_cache=Elimina les versions antigues
 arch.config.clean_cache.tip=Si cal eliminar les versions antigues d'un paquet emmagatzemat al disc durant la desinstal·lació
 arch.config.check_dependency_breakage=Check dependency version breakage

bauh/gems/arch/resources/locale/de

@@ -41,6 +41,8 @@ arch.config.aur_build_only_chosen=Build only chosen (AUR)
 arch.config.aur_build_only_chosen.tip=Some AUR packages have a common PKGBUILD shared with other packages and that defines build instructions for each one. This property enabled will ensure that only the chosen package will be built.
 arch.config.aur_remove_build_dir=Remove build directory (AUR)
 arch.config.aur_remove_build_dir.tip=If a package's generated build directory should be removed after the operations is finished.
+arch.config.categories_exp=Categories expiration
+arch.config.categories_exp.tip=It defines the expiration time (in HOURS) of the packages categories mapping file stored in disc. Use 0 so that it is always updated during initialization.
 arch.config.clean_cache=Remove old versions
 arch.config.clean_cache.tip=Whether old versions of a package stored on disk should be removed during uninstall
 arch.config.check_dependency_breakage=Check dependency version breakage

bauh/gems/arch/resources/locale/en

@@ -41,6 +41,8 @@ arch.config.aur_build_only_chosen=Build only chosen (AUR)
 arch.config.aur_build_only_chosen.tip=Some AUR packages have a common PKGBUILD shared with other packages and that defines build instructions for each one. This property enabled will ensure that only the chosen package will be built.
 arch.config.aur_remove_build_dir=Remove build directory (AUR)
 arch.config.aur_remove_build_dir.tip=If a package's generated build directory should be removed after the operations is finished.
+arch.config.categories_exp=Categories expiration
+arch.config.categories_exp.tip=It defines the expiration time (in HOURS) of the packages categories mapping file stored in disc. Use 0 so that it is always updated during initialization.
 arch.config.clean_cache=Remove old versions
 arch.config.clean_cache.tip=Whether old versions of a package stored on disk should be removed during uninstall
 arch.config.check_dependency_breakage=Check dependency version breakage

bauh/gems/arch/resources/locale/es

@@ -41,6 +41,8 @@ arch.config.aur_build_only_chosen=Compilar solo elegido (AUR)
 arch.config.aur_build_only_chosen.tip=Algunos paquetes AUR tienen un PKGBUILD común compartido con otros paquetes y que define las instrucciones de construcción para cada uno. Esta propiedad habilitada garantizará que solo se compile el paquete elegido.
 arch.config.aur_remove_build_dir=Eliminar directorio de compilación (AUR)
 arch.config.aur_remove_build_dir.tip=Si el directorio de compilación generado para un paquete debe ser eliminado una vez finalizada la operación.
+arch.config.categories_exp=Expiración de categorías
+arch.config.categories_exp.tip=Define el tiempo de vencimiento (en HORAS) del archivo de mapeo de categorías de paquetes almacenado en el disco. Utilice 0 para que siempre se actualice durante la inicialización.
 arch.config.clean_cache=Eliminar versiones antiguas
 arch.config.clean_cache.tip=Si las versiones antiguas de un paquete almacenado en el disco deben ser eliminadas durante la desinstalación
 arch.config.check_dependency_breakage=Verificar rotura de versión de dependencia

bauh/gems/arch/resources/locale/fr

@@ -41,6 +41,8 @@ arch.config.aur_build_only_chosen=Compiler uniquement la sélection (AUR)
 arch.config.aur_build_only_chosen.tip=Certains paquets AUR ont un PKGBUILD commun partagé avec d'autres packages et ça définit les insctructions de compilation pour chacun d'eux. Cette propriété assure que le paquet sélectionné sera le seul compilé.
 arch.config.aur_remove_build_dir=Supprimer le répertoire de compilation (AUR)
 arch.config.aur_remove_build_dir.tip=Si le repertoire de compilation généré doit être supprimé à la fin des opérations.
+arch.config.categories_exp=Categories expiration
+arch.config.categories_exp.tip=It defines the expiration time (in HOURS) of the packages categories mapping file stored in disc. Use 0 so that it is always updated during initialization.
 arch.config.clean_cache=Supprimer les anciennes versions
 arch.config.clean_cache.tip=Supprimer les vielles versions d'un package stocké sur le disque pendant la désinstallation
 arch.config.check_dependency_breakage=Vérification pour éviter qu'une version de casse les dépendances

bauh/gems/arch/resources/locale/it

@@ -41,6 +41,8 @@ arch.config.aur_build_only_chosen=Build only chosen (AUR)
 arch.config.aur_build_only_chosen.tip=Some AUR packages have a common PKGBUILD shared with other packages and that defines build instructions for each one. This property enabled will ensure that only the chosen package will be built.
 arch.config.aur_remove_build_dir=Remove build directory (AUR)
 arch.config.aur_remove_build_dir.tip=If a package's generated build directory should be removed after the operations is finished.
+arch.config.categories_exp=Categories expiration
+arch.config.categories_exp.tip=It defines the expiration time (in HOURS) of the packages categories mapping file stored in disc. Use 0 so that it is always updated during initialization.
 arch.config.clean_cache=Rimuovi le vecchie versioni
 arch.config.clean_cache.tip=Se le vecchie versioni di un pacchetto memorizzate sul disco devono essere rimosse durante la disinstallazione
 arch.config.check_dependency_breakage=Check dependency version breakage

bauh/gems/arch/resources/locale/pt

@@ -41,6 +41,8 @@ arch.config.aur_build_only_chosen=Construir somente escolhido (AUR)
 arch.config.aur_build_only_chosen.tip=Alguns pacotes do AUR têm um PKGBUILD comum a outros pacotes e que define a construção para todos. Essa propriedade ativada garantirá que somente o pacote escolhido será construído.
 arch.config.aur_remove_build_dir=Remover diretório de construção (AUR)
 arch.config.aur_remove_build_dir.tip=Se o diretório gerado para a construção de um pacote do AUR deve ser removido após a operação ser finalizada.
+arch.config.categories_exp=Expiração de categorias
+arch.config.categories_exp.tip=Define o tempo de expiração (em HORAS) do arquivo de mapeamento de categorias de pacotes armazenado em disco. Utilize 0 para que ele seja sempre atualizado durante a inicialização.
 arch.config.clean_cache=Remover versões antigas
 arch.config.clean_cache.tip=Se versões antigas de um pacote armazenadas em disco devem ser removidas durante a desinstalação
 arch.config.check_dependency_breakage=Verificar quebra de dependência de versão

bauh/gems/arch/resources/locale/ru

@@ -41,6 +41,8 @@ arch.config.aur_build_only_chosen=Build only chosen (AUR)
 arch.config.aur_build_only_chosen.tip=Some AUR packages have a common PKGBUILD shared with other packages and that defines build instructions for each one. This property enabled will ensure that only the chosen package will be built.
 arch.config.aur_remove_build_dir=Remove build directory (AUR)
 arch.config.aur_remove_build_dir.tip=If a package's generated build directory should be removed after the operations is finished.
+arch.config.categories_exp=Categories expiration
+arch.config.categories_exp.tip=It defines the expiration time (in HOURS) of the packages categories mapping file stored in disc. Use 0 so that it is always updated during initialization.
 arch.config.clean_cache=Remove old versions
 arch.config.clean_cache.tip=Whether old versions of a package stored on disk should be removed during uninstall
 arch.config.check_dependency_breakage=Check dependency version breakage

bauh/gems/arch/resources/locale/tr

@@ -41,6 +41,8 @@ arch.config.aur_build_only_chosen=Build only chosen (AUR)
 arch.config.aur_build_only_chosen.tip=Some AUR packages have a common PKGBUILD shared with other packages and that defines build instructions for each one. This property enabled will ensure that only the chosen package will be built.
 arch.config.aur_remove_build_dir=Remove build directory (AUR)
 arch.config.aur_remove_build_dir.tip=If a package's generated build directory should be removed after the operations is finished.
+arch.config.categories_exp=Categories expiration
+arch.config.categories_exp.tip=It defines the expiration time (in HOURS) of the packages categories mapping file stored in disc. Use 0 so that it is always updated during initialization.
 arch.config.clean_cache=Önbelleği temizle
 arch.config.clean_cache.tip=Disk üzerinde kurulu bir paketin eski sürümlerinin kaldırma sırasında kaldırılıp kaldırılmayacağı
 arch.config.check_dependency_breakage=Check dependency version breakage

bauh/gems/snap/config.py

@@ -3,5 +3,5 @@ from bauh.gems.snap import CONFIG_FILE
 
 
 def read_config(update_file: bool = False) -> dict:
-    template = {'install_channel': False}
+    template = {'install_channel': False, 'categories_exp': 24}
     return read(CONFIG_FILE, template, update_file=update_file)

bauh/gems/snap/controller.py

@@ -11,7 +11,7 @@ from bauh.api.abstract.handler import ProcessWatcher, TaskManager
 from bauh.api.abstract.model import SoftwarePackage, PackageHistory, PackageUpdate, PackageSuggestion, \
     SuggestionPriority, CustomSoftwareAction, PackageStatus
 from bauh.api.abstract.view import SingleSelectComponent, SelectViewType, InputOption, ViewComponent, PanelComponent, \
-    FormComponent
+    FormComponent, TextInputComponent
 from bauh.api.exception import NoInternetException
 from bauh.commons import resource
 from bauh.commons.category import CategoriesDownloader
@@ -19,7 +19,7 @@ from bauh.commons.config import save_config
 from bauh.commons.html import bold
 from bauh.commons.system import SystemProcess, ProcessHandler, new_root_subprocess, get_human_size_str
 from bauh.commons.view_utils import new_select
-from bauh.gems.snap import snap, URL_CATEGORIES_FILE, SNAP_CACHE_PATH, CATEGORIES_FILE_PATH, SUGGESTIONS_FILE, \
+from bauh.gems.snap import snap, URL_CATEGORIES_FILE, CATEGORIES_FILE_PATH, SUGGESTIONS_FILE, \
     get_icon_path, snapd, CONFIG_FILE, ROOT_DIR
 from bauh.gems.snap.config import read_config
 from bauh.gems.snap.model import SnapApplication
@@ -285,7 +285,7 @@ class SnapManager(SoftwareManager):
 
     def _start_category_task(self, task_man: TaskManager):
         if task_man:
-            task_man.register_task('snap_cats', self.i18n['task.download_categories'].format('Snap'), get_icon_path())
+            task_man.register_task('snap_cats', self.i18n['task.download_categories'], get_icon_path())
             task_man.update_progress('snap_cats', 50, None)
 
     def _finish_category_task(self, task_man: TaskManager):
@@ -294,11 +294,14 @@ class SnapManager(SoftwareManager):
             task_man.finish_task('snap_cats')
 
     def prepare(self, task_manager: TaskManager, root_password: str, internet_available: bool):
-        Thread(target=read_config, args=(True,), daemon=True).start()
+        snap_config = read_config()
 
         CategoriesDownloader(id_='snap', manager=self, http_client=self.http_client, logger=self.logger,
-                             url_categories_file=URL_CATEGORIES_FILE, disk_cache_dir=SNAP_CACHE_PATH,
+                             url_categories_file=URL_CATEGORIES_FILE,
                              categories_path=CATEGORIES_FILE_PATH,
+                             expiration=snap_config['categories_exp'] if isinstance(snap_config['categories_exp'], int) else 0,
+                             internet_connection=internet_available,
+                             internet_checker=self.context.internet_checker,
                              before=lambda: self._start_category_task(task_manager),
                              after=lambda: self._finish_category_task(task_manager)).start()
 
@@ -434,20 +437,31 @@ class SnapManager(SoftwareManager):
 
     def get_settings(self, screen_width: int, screen_height: int) -> ViewComponent:
         snap_config = read_config()
+        max_width = 200
 
         install_channel = new_select(label=self.i18n['snap.config.install_channel'],
                                      opts=[(self.i18n['yes'].capitalize(), True, None),
                                            (self.i18n['no'].capitalize(), False, None)],
                                      value=bool(snap_config['install_channel']),
-                                     id_='install_channel',
+                                     id_='snap_install_channel',
-                                     max_width=200,
+                                     max_width=max_width,
                                      tip=self.i18n['snap.config.install_channel.tip'])
 
-        return PanelComponent([FormComponent([install_channel], self.i18n['installation'].capitalize())])
+        categories_exp = TextInputComponent(id_='snap_cat_exp',
+                                            value=snap_config['categories_exp'] if isinstance(snap_config['categories_exp'], int) else '',
+                                            max_width=max_width,
+                                            only_int=True,
+                                            label=self.i18n['snap.config.categories_exp'],
+                                            tooltip=self.i18n['snap.config.categories_exp.tip'])
+
+        return PanelComponent([FormComponent([install_channel, categories_exp], self.i18n['installation'].capitalize())])
 
     def save_settings(self, component: ViewComponent) -> Tuple[bool, Optional[List[str]]]:
         config = read_config()
-        config['install_channel'] = component.components[0].components[0].get_selected()
+
+        panel = component.components[0]
+        config['install_channel'] = panel.get_component('snap_install_channel').get_selected()
+        config['categories_exp'] = panel.get_component('snap_cat_exp').get_int_value()
 
         try:
             save_config(config, CONFIG_FILE)

bauh/gems/snap/resources/locale/ca

@@ -6,6 +6,8 @@ snap.action.channel.status=Changing channel of
 snap.action.refresh.confirm=Actualitza {} ?
 snap.action.refresh.label=Actualitza
 snap.action.refresh.status=S’està actualitzant
+snap.config.categories_exp=Categories expiration
+snap.config.categories_exp.tip=It defines the expiration time (in HOURS) of the Snaps categories mapping file stored in disc. Use 0 so that it is always updated during initialization.
 snap.config.install_channel=Display channels
 snap.config.install_channel.tip=It allows to select one of the available channels for the given application to be installed
 snap.info.channel=channel

bauh/gems/snap/resources/locale/de

@@ -6,6 +6,8 @@ snap.action.channel.status=Changing channel of
 snap.action.refresh.confirm=Erneuern {} ?
 snap.action.refresh.label=Erneuern
 snap.action.refresh.status=Erneuern
+snap.config.categories_exp=Categories expiration
+snap.config.categories_exp.tip=It defines the expiration time (in HOURS) of the Snaps categories mapping file stored in disc. Use 0 so that it is always updated during initialization.
 snap.config.install_channel=Display channels
 snap.config.install_channel.tip=It allows to select one of the available channels for the given application to be installed
 snap.info.channel=channel

bauh/gems/snap/resources/locale/en

@@ -6,6 +6,8 @@ snap.action.channel.status=Changing channel of
 snap.action.refresh.confirm=Refresh {} ?
 snap.action.refresh.label=Refresh
 snap.action.refresh.status=Refreshing
+snap.config.categories_exp=Categories expiration
+snap.config.categories_exp.tip=It defines the expiration time (in HOURS) of the Snaps categories mapping file stored in disc. Use 0 so that it is always updated during initialization.
 snap.config.install_channel=Display channels
 snap.config.install_channel.tip=It allows to select one of the available channels for the given application to be installed
 snap.info.channel=channel

bauh/gems/snap/resources/locale/es

@@ -6,6 +6,8 @@ snap.action.channel.label=Cambiar canal
 snap.action.refresh.confirm=Actualizar {} ?
 snap.action.refresh.label=Actualizar
 snap.action.refresh.status=Actualizando
+snap.config.categories_exp=Expiración de categorías
+snap.config.categories_exp.tip=Define el tiempo de vencimiento (en HORAS) del archivo de mapeo de categorías de Snaps almacenado en el disco. Utilice 0 para que siempre se actualice durante la inicialización.
 snap.config.install_channel=Mostrar canales
 snap.config.install_channel.tip=Permite seleccionar uno de los canales disponibles para la aplicación a instalar
 snap.info.channel=canal

bauh/gems/snap/resources/locale/fr

@@ -6,6 +6,8 @@ snap.action.channel.status=Changement de canal de
 snap.action.refresh.confirm=Actualiser {} ?
 snap.action.refresh.label=Actualiser
 snap.action.refresh.status=Actualisation
+snap.config.categories_exp=Categories expiration
+snap.config.categories_exp.tip=It defines the expiration time (in HOURS) of the Snaps categories mapping file stored in disc. Use 0 so that it is always updated during initialization.
 snap.config.install_channel=Afficher les canaux
 snap.config.install_channel.tip=Permet de sélectionner un des canaux disponibles pour l'application à installer
 snap.info.channel=canal

bauh/gems/snap/resources/locale/it

@@ -6,6 +6,8 @@ snap.action.channel.status=Changing channel of
 snap.action.refresh.confirm=Ripristina {} ?
 snap.action.refresh.label=Ripristina
 snap.action.refresh.status=Ripristinare
+snap.config.categories_exp=Categories expiration
+snap.config.categories_exp.tip=It defines the expiration time (in HOURS) of the Snaps categories mapping file stored in disc. Use 0 so that it is always updated during initialization.
 snap.config.install_channel=Display channels
 snap.config.install_channel.tip=It allows to select one of the available channels for the given application to be installed
 snap.info.channel=channel

bauh/gems/snap/resources/locale/pt

@@ -6,6 +6,8 @@ snap.action.channel.status=Alterando canal de
 snap.action.refresh.confirm=Atualizar {} ?
 snap.action.refresh.label=Atualizar
 snap.action.refresh.status=Atualizando
+snap.config.categories_exp=Expiração de categorias
+snap.config.categories_exp.tip=Define o tempo de expiração (em HORAS) do arquivo de mapeamento de categorias Snap armazenado em disco. Utilize 0 para que ele seja sempre atualizado durante a inicialização.
 snap.config.install_channel=Exibir canais
 snap.config.install_channel.tip=Permite-se selecionar um dos canais disponíveis para o aplicativo a ser instalado
 snap.info.channel=canal

bauh/gems/snap/resources/locale/ru

@@ -6,6 +6,8 @@ snap.action.channel.status=Changing channel of
 snap.action.refresh.confirm=Обновить {} ?
 snap.action.refresh.label=Обновить
 snap.action.refresh.status=Обновляется
+snap.config.categories_exp=Categories expiration
+snap.config.categories_exp.tip=It defines the expiration time (in HOURS) of the Snaps categories mapping file stored in disc. Use 0 so that it is always updated during initialization.
 snap.config.install_channel=Display channels
 snap.config.install_channel.tip=It allows to select one of the available channels for the given application to be installed
 snap.info.channel=channel

bauh/gems/snap/resources/locale/tr

@@ -6,6 +6,8 @@ snap.action.channel.status=Changing channel of
 snap.action.refresh.confirm=Yenile {} ?
 snap.action.refresh.label=Yenile
 snap.action.refresh.status=Yenileniyor
+snap.config.categories_exp=Categories expiration
+snap.config.categories_exp.tip=It defines the expiration time (in HOURS) of the Snaps categories mapping file stored in disc. Use 0 so that it is always updated during initialization.
 snap.config.install_channel=Display channels
 snap.config.install_channel.tip=It allows to select one of the available channels for the given application to be installed
 snap.info.channel=channel

bauh/view/core/controller.py

@@ -400,6 +400,8 @@ class GenericSoftwareManager(SoftwareManager):
                 return man.requires_root(action, app)
 
     def prepare(self, task_manager: TaskManager, root_password: str, internet_available: bool):
+        ti = time.time()
+        self.logger.info("Initializing")
         if self.managers:
             internet_on = self.context.is_internet_available()
             taskman = task_manager if task_manager else TaskManager()  # empty task manager to prevent null pointers
@@ -408,6 +410,9 @@ class GenericSoftwareManager(SoftwareManager):
                     man.prepare(taskman, root_password, internet_on)
                     self._already_prepared.append(man)
 
+        tf = time.time()
+        self.logger.info("Finished. Took {0:.2f} seconds".format(tf - ti))
+
     def cache_available_managers(self):
         if self.managers:
             for man in self.managers:

bauh/view/resources/locale/ca

@@ -391,7 +391,7 @@ status.caching_data=S’estan emmagatzemant {} dades a la memòria cau al disc
 style=estil
 success=success
 summary=resum
-task.download_categories=Download de categories [ {} ]
+task.download_categories=Download de categories
 task.waiting_task=Waiting for {}
 type=tipus
 uninstall=desinstal·la

bauh/view/resources/locale/de

@@ -391,7 +391,7 @@ status.caching_data={} Daten auf der Festplatte zwischenspeichern
 style=Stil
 success=success
 summary=Zusammenfassung
-task.download_categories=Downloading categories [ {} ]
+task.download_categories=Downloading categories
 task.waiting_task=Waiting for {}
 type=Typ
 uninstall=Deinstallation

bauh/view/resources/locale/en

@@ -392,7 +392,7 @@ status.caching_data=Caching {} data to disk
 style=style
 success=success
 summary=summary
-task.download_categories=Downloading categories [ {} ]
+task.download_categories=Downloading categories
 task.waiting_task=Waiting for {}
 type=type
 uninstall=uninstall

bauh/view/resources/locale/es

@@ -394,7 +394,7 @@ status.caching_data=Almacenando en antememoria los datos de {} para el disco
 style=estilo
 success=éxito
 summary=resumen
-task.download_categories=Descargando categorías [ {} ]
+task.download_categories=Descargando categorías
 task.waiting_task=Esperando por {}
 type=tipo
 uninstall=desinstalar

bauh/view/resources/locale/fr

@@ -388,7 +388,7 @@ status.caching_data=Mise en cache des données de {} sur disque
 style=style
 success=succès
 summary=résumé
-task.download_categories=Téléchargement des catégories [ {} ]
+task.download_categories=Téléchargement des catégories
 task.waiting_task=Waiting for {}
 type=type
 uninstall=désinstaller

bauh/view/resources/locale/it

@@ -394,7 +394,7 @@ status.caching_data=Memorizza {} dati sul disco
 style=stile
 success=success
 summary=riepilogo
-task.download_categories=Download delle categorie [ {} ]
+task.download_categories=Download delle categorie
 task.waiting_task=Waiting for {}
 type=tipe
 uninstall=Disinstalla

bauh/view/resources/locale/pt

@@ -392,7 +392,7 @@ status.caching_data=Cacheando dados de {} para o disco
 style=estilo
 success=sucesso
 summary=resumo
-task.download_categories=Baixando categorias [ {} ]
+task.download_categories=Baixando categorias
 task.waiting_task=Aguardando {}
 type=tipo
 uninstall=desinstalar

bauh/view/resources/locale/ru

@@ -391,7 +391,7 @@ status.caching_data=Кэширование данных {} на диск
 style=Стиль
 success=success
 summary=Суммарно
-task.download_categories=Downloading categories [ {} ]
+task.download_categories=Downloading categories
 task.waiting_task=Waiting for {}
 type=Тип
 uninstall=Деинсталляция

bauh/view/resources/locale/tr

@@ -391,7 +391,7 @@ status.caching_data={} verilerini diske önbellekle
 style=Görünüm biçimi
 success=başarılı
 summary=özet
-task.download_categories=Kategoriler indiriliyor [ {} ]
+task.download_categories=Kategoriler indiriliyor
 task.waiting_task=Waiting for {}
 type=tür
 uninstall=kaldır