ICode9

精准搜索请尝试: 精确搜索
首页 > 其他分享> 文章详细

django使用celery及踩坑

2022-03-20 16:03:17  阅读:230  来源: 互联网

标签:settings app django celery 使用 contrib import


项目结构

在这里插入图片描述

项目依赖

celery==5.1.2   
Django==3.2.12  
django-celery-beat==2.2.1  
django-celery-results==2.2.0  
django-redis==5.2.0  
eventlet==0.33.0  
greenlet==1.1.2   
redis==4.1.4  

安装package

  • 安装celery
pip install celery
  • 安装django-celery-results
    用于保存celery任务结果
pip install celery
  • 安装django-celery-beat
    用于执行周期任务
pip install celery
  • 安装eventlet
    windows环境下需要
pip install eventlet

在django项目中使用

  • 在项目设置文件settings.py中配置celery
    settings.py
"""  
Django settings for demo project.  
  
Generated by 'django-admin startproject' using Django 3.2.12.  
  
For more information on this file, see  
https://docs.djangoproject.com/en/3.2/topics/settings/  
  
For the full list of settings and their values, see  
https://docs.djangoproject.com/en/3.2/ref/settings/  
"""  
  
from pathlib import Path  
  
# Build paths inside the project like this: BASE_DIR / 'subdir'.  
BASE_DIR = Path(__file__).resolve().parent.parent  
  
# Quick-start development settings - unsuitable for production  
# See https://docs.djangoproject.com/en/3.2/howto/deployment/checklist/  
  
# SECURITY WARNING: keep the secret key used in production secret!  
SECRET_KEY = 'django-insecure-%8q+wpy=)*@237)o#!pgkn&^$qr9-h!w12jociuv4i^f#&z70d'  
  
# SECURITY WARNING: don't run with debug turned on in production!  
DEBUG = True  
  
ALLOWED_HOSTS = []  
  
# Application definition  
  
INSTALLED_APPS = [  
    'django.contrib.admin',  
 'django.contrib.auth',  
 'django.contrib.contenttypes',  
 'django.contrib.sessions',  
 'django.contrib.messages',  
 'django.contrib.staticfiles',  
 'app1',  #注册应用
 'django_celery_results',  #用于保存celery任务结果
 'django_celery_beat',  #用于执行celery周期任务
]  
  
MIDDLEWARE = [  
    'django.middleware.security.SecurityMiddleware',  
 'django.contrib.sessions.middleware.SessionMiddleware',  
 'django.middleware.common.CommonMiddleware',  
 'django.middleware.csrf.CsrfViewMiddleware',  
 'django.contrib.auth.middleware.AuthenticationMiddleware',  
 'django.contrib.messages.middleware.MessageMiddleware',  
 'django.middleware.clickjacking.XFrameOptionsMiddleware',  
]  
  
ROOT_URLCONF = 'demo.urls'  
  
TEMPLATES = [  
    {  
        'BACKEND': 'django.template.backends.django.DjangoTemplates',  
 'DIRS': [BASE_DIR / 'templates']  
        ,  
 'APP_DIRS': True,  
 'OPTIONS': {  
            'context_processors': [  
                'django.template.context_processors.debug',  
 'django.template.context_processors.request',  
 'django.contrib.auth.context_processors.auth',  
 'django.contrib.messages.context_processors.messages',  
 ],  
 },  
 },  
]  
  
WSGI_APPLICATION = 'demo.wsgi.application'  
  
# Database  
# https://docs.djangoproject.com/en/3.2/ref/settings/#databases  
  
DATABASES = {  
    'default': {  
        'ENGINE': 'django.db.backends.sqlite3',  
 'NAME': BASE_DIR / 'db.sqlite3',  
 }  
}  
  
# Password validation  
# https://docs.djangoproject.com/en/3.2/ref/settings/#auth-password-validators  
  
AUTH_PASSWORD_VALIDATORS = [  
    {  
        'NAME': 'django.contrib.auth.password_validation.UserAttributeSimilarityValidator',  
 },  
 {  
        'NAME': 'django.contrib.auth.password_validation.MinimumLengthValidator',  
 },  
 {  
        'NAME': 'django.contrib.auth.password_validation.CommonPasswordValidator',  
 },  
 {  
        'NAME': 'django.contrib.auth.password_validation.NumericPasswordValidator',  
 },  
]  
  
# Internationalization  
# https://docs.djangoproject.com/en/3.2/topics/i18n/  
  
LANGUAGE_CODE = 'zh-hans'  #语言修改为中文
  
TIME_ZONE = 'Asia/Shanghai'  #时区修改为上海
  
USE_I18N = True  
  
USE_L10N = True  
  
USE_TZ = False  
  
# Static files (CSS, JavaScript, Images)  
# https://docs.djangoproject.com/en/3.2/howto/static-files/  
  
STATIC_URL = '/static/'  
  
# Default primary key field type  
# https://docs.djangoproject.com/en/3.2/ref/settings/#default-auto-field  
  
DEFAULT_AUTO_FIELD = 'django.db.models.BigAutoField'  
  
# celery配置  
# 任务结果使用数据库来储存  
CELERY_RESULT_BACKEND = 'django-db'  
# celery 缓存
CELERY_CACHE_BACKEND = 'django-cache'

# 用了存放消息队列的位置,此处使用redis,默认使用rabbitmq
CELERY_BROKER_URL = 'redis://127.0.0.1:6379/2'  
  
#: Only add pickle to this list if your broker is secured  
#: from unwanted access (see userguide/security.html)  
CELERY_ACCEPT_CONTENT = ['json']  
CELERY_TASK_SERIALIZER = 'json'  
  
# 缓存配置  
CACHES = {  
    'default': {  
        'BACKEND': 'django_redis.cache.RedisCache',  
 'LOCATION': 'redis://127.0.0.1:6379/0',  
 },  
 'celery': {  
        'BACKEND': 'django_redis.cache.RedisCache',  
 'LOCATION': 'redis://127.0.0.1:6379/1',  
 },  
}
  • 在项目同名目录下创建celery.py文件
    celery.py
from __future__ import absolute_import, unicode_literals  
import os  
from celery import Celery  
from celery.schedules import timedelta  
# set the default Django settings module for the 'celery' program.  
os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'demo.settings')  
  
app = Celery('demo')  
app.conf.timezone = 'Asia/Shanghai'  
app.conf.enable_utc = False  
  
# Using a string here means the worker doesn't have to serialize  
# the configuration object to child processes.  
# - namespace='CELERY' means all celery-related configuration keys  
#   should have a `CELERY_` prefix.  
app.config_from_object('django.conf:settings', namespace='CELERY')  
# 周期任务配置
app.conf.beat_schedule = {  
    'add-every-10-seconds': {  
        'task': 'app1.tasks.func2',  
 'schedule': timedelta(seconds=10),  
 'args': ()  
    },  
}  
# Load task modules from all registered Django app configs.  
app.autodiscover_tasks()
  • 在项目同名目录下的__init__.py中作如下配置,这样可以确保在Django启动时加载应用程序,以便@shared_task装饰器将使用该应用程序
    __init__py
from .celery import app as celery_app  
  
__all__ = ('celery_app',)
  • 在app目录下新建tasks.py文件
    task.py
from celery import shared_task  
  
  
@shared_task  
def func1():  
    return "异步任务"  
  
  
@shared_task  
def func2():  
    return "周期任务"
  • 任务调用:异步任务为主动触发,定时和周期任务需有beat分发任务,worker去执行
    views.py
from django.http import JsonResponse  
from .tasks import func1  
  
  
# Create your views here.  
  
def async_task(request):  
    func1.delay()  
    return JsonResponse({"message": "这是celery异步任务!"},json_dumps_params={"ensure_ascii":False})

项目运行

  • 启动django项目
python manage.py runserver 127.0.0.1:8000
  • 启动celery beat(当需要执行定时任务时需要启动beat)
celery -A demo beat -l INFO

在这里插入图片描述

  • 正确启动 celery worker
celery -A demo worker -l INFO --pool=solo

在这里插入图片描述

踩坑

  • 不建议使用django-celery,容易出现很多包版本配合的问题
  • 使用 celery -A demo worker -l INFO 启动worker会出现
ValueError: not enough values to unpack (expected 3, got 0)

在这里插入图片描述

  • 使用 celery -A demo worker -l INFO -P eventlet 启动worker会出现
django.db.utils.DatabaseError: DatabaseWrapper objects created in a thread can only be used in that same thread

在这里插入图片描述

标签:settings,app,django,celery,使用,contrib,import
来源: https://blog.csdn.net/qq_29983177/article/details/123614701

本站声明: 1. iCode9 技术分享网(下文简称本站)提供的所有内容,仅供技术学习、探讨和分享;
2. 关于本站的所有留言、评论、转载及引用,纯属内容发起人的个人观点,与本站观点和立场无关;
3. 关于本站的所有言论和文字,纯属内容发起人的个人观点,与本站观点和立场无关;
4. 本站文章均是网友提供,不完全保证技术分享内容的完整性、准确性、时效性、风险性和版权归属;如您发现该文章侵犯了您的权益,可联系我们第一时间进行删除;
5. 本站为非盈利性的个人网站,所有内容不会用来进行牟利,也不会利用任何形式的广告来间接获益,纯粹是为了广大技术爱好者提供技术内容和技术思想的分享性交流网站。

专注分享技术,共同学习,共同进步。侵权联系[81616952@qq.com]

Copyright (C)ICode9.com, All Rights Reserved.

ICode9版权所有