ICode9
精准搜索请尝试： 精确搜索
首页
编程语言>
数据库
系统相关
互联网
其他分享
Python
Java
JavaScript
android
PHP
首页 > 其他分享> 文章详细

dubbo泛化调用踩坑精度问题

2022-06-02 10:31:15  阅读：162  来源： 互联网

标签：dubbo 调用 泛化 Object null class String


问题背景 开放平台对接外部服务暴露http接口，然后http接口请求根据参数将请求分发至内部dubbo服务，分发动作使用的dubbo泛化调用。计费接口测试时发现cost字段（BigDecimal）出现精度问题 测试数据的cost值为0.01 http网关服务日志，可以看到入参的cost=0.01，没有问题 
2021-04-22 10:40:18,597 INFO [4fb5b4087d2c462eab8b9ede87b8d272] [DubboServerHandler-10.200.25.210:16976-thread-477] com.dianwoda.open.toolbox.dubbo.filter.DubboInvokeLogFilter:invoke:67 Invoke com.alibaba.dubbo.rpc.service.GenericService.$invoke(java.lang.String,[Ljava.lang.String;,[Ljava.lang.Object;):1.0.0 cost 21ms , 10.200.25.210:0=>10.200.36.21:16815 , arguments=["addBalanceLogCheckTradeNoStr",["com.dianwoda.billing.settle.provider.outmoded.dto.BalanceDTO"],[{"cityId":1,"riderId":5150213,"riderType":0,"type":95,"cost":0.01,"tradeNoStr":"6899423540852876801619059218000"}]] , result=true

dubbo服务端日志，可以看到如惨已经出现问题：“cost”:0.009999999776482582

2021-04-22 10:40:18,577 INFO [4fb5b4087d2c462eab8b9ede87b8d272] [DubboServerHandler-10.200.36.21:16815-thread-498] dubbo.accesslog.com.....billing.settle.provider.outmoded.RiderTradeCostSettleOutmodedProvider:info:42  [DUBBO] [2021-04-22 10:40:18] 10.200.25.210:42688 -> 10.200.36.21:16815 - com.....billing.settle.provider.outmoded.RiderTradeCostSettleOutmodedProvider:1.0.0 addBalanceLogCheckTradeNoStr(com.....billing.settle.provider.outmoded.dto.BalanceDTO) [{"reason":null,"bankCard":null,"withdrawType":null,"bankName":null,"cityId":1,"type":95,"riderId":5150213,"riskChecked":null,"payType":null,"feature":null,"blocked":null,"id":null,"riderType":0,"batchRecordId":null,"batchNo":null,"cost":0.009999999776482582,"tradeNo":null,"bankCardType":null,"currentServiceType":null,"effectiveBalance":null,"tradeWay":null,"sourceTradeNo":null,"sourceBalanceType":null,"factorage":null,"verifyTm":null,"finishTm":null,"tradeNoStr":"6899423540852876801619059218000","paid":null,"name":null,"insTm":null,"withdrawTm":null,"account":null}], dubbo version: 2.5.3, current host: 10.200.36.21

问题排查

泛化调用的锅？

网关侧封装的泛化调用代码，代码看到此处，精度问题有两种可能原因

  1. dubbo泛化调用中类型转换问题？
  2. 泛化调用前参数转换问题？
public ResponseDTO<String> dock(DockRequest request) {
    Object retObj;
    try {
        CtURL url = CtURL.parseURL(request.getRequestType());
        GenericService genericService = getDubboGenericService(url);
        ...
        Class clazz = getClass(url);
        // 上下文类加载器中存在类则尝试通过上下文获取bean进行调用
        if (null != clazz) {
            Map<String, Object> beanMap = applicationContext.getBeansOfType(clazz);
            if (beanMap.size() == 1) {
                retObj = invokeMethod(clazz, beanMap.values().iterator().next(), url,
                        request.getRequestBody());
            } else {
                // 上下文同类型bean存在多个走泛化调用
                retObj = genericService
                        .$invoke(url.getMethod(), parameterTypes, convertArray(jsonArray));
            }
        } else {
            // 类加载器中不存在class直接泛化调用
            retObj = genericService
                    .$invoke(url.getMethod(), parameterTypes, convertArray(jsonArray));
        }
    }...
}
// 注册dubbo泛化调用消费者至spring上下文
private GenericService getDubboGenericService(CtURL url) {
    String group = url.getGroup();
    String version = url.getVersion();
    String beanName =
            StringUtils.firstNonBlank(url.getGroup(), "DEFAULT") + "/" + url.getService() + (
                    StringUtils.isNotBlank(version) ? "" : "/" + version);

    if (applicationContext.containsBean(beanName)) {
        return (GenericService) applicationContext.getBean(beanName);
    }
    synchronized (applicationContext) {
        if (applicationContext.containsBean(beanName)) {
            return (GenericService) applicationContext.getBean(beanName);
        }

        AbstractBeanDefinition definition = BeanDefinitionBuilder
                .genericBeanDefinition(ReferenceBean.class)
                .addPropertyValue("application", new ApplicationConfig("dock-common"))
                .addPropertyValue("registries", registryConfigs)
                .addPropertyValue("interface", url.getService())
                .addPropertyValue("group", group).addPropertyValue("version", version)
                .addPropertyValue("generic", "true").addPropertyValue("retries", 0)
                .addPropertyValue("timeout", 3000).getBeanDefinition();

        BeanFactory beanFactory = applicationContext.getAutowireCapableBeanFactory();
        BeanDefinitionRegistry registry = (BeanDefinitionRegistry) beanFactory;
        registry.registerBeanDefinition(beanName, definition);
        return (GenericService) beanFactory.getBean(beanName);
    }
}

泛化调用原理

dubbo消费者调用服务提供者过程不多说，消费者与提供者端invoker都会经历对应group的filter，泛化调用则由其中一个filter实现：com.alibaba.dubbo.rpc.filter.GenericImplFilter，将invocation方法设置为com.alibaba.dubbo.common.Constants#$INVOKE，提供者端则由com.alibaba.dubbo.rpc.filter.GenericFilter责任链节点识别泛化调用，消费者端GenericImplFilter源码如下

public Result invoke(Invoker<?> invoker, Invocation invocation) throws RpcException {
    String generic = invoker.getUrl().getParameter(Constants.GENERIC_KEY);
    // 确定是泛化调用
    if (ProtocolUtils.isGeneric(generic)
            && !Constants.$INVOKE.equals(invocation.getMethodName())
            && invocation instanceof RpcInvocation) {
        RpcInvocation invocation2 = (RpcInvocation) invocation;
        String methodName = invocation2.getMethodName();
        Class<?>[] parameterTypes = invocation2.getParameterTypes();
        Object[] arguments = invocation2.getArguments();
        String[] types = new String[parameterTypes.length];
        for (int i = 0; i < parameterTypes.length; i++) {
            types[i] = ReflectUtils.getName(parameterTypes[i]);
        }
        Object[] args;
        // generic参数为bean，则以bean的方式序列化参数
        if (ProtocolUtils.isBeanGenericSerialization(generic)) {
            args = new Object[arguments.length];
            for (int i = 0; i < arguments.length; i++) {
                args[i] = JavaBeanSerializeUtil.serialize(arguments[i], JavaBeanAccessor.METHOD);
            }
        } else {
            // 否则使用工具类序列化
            args = PojoUtils.generalize(arguments);
        }
        invocation2.setMethodName(Constants.$INVOKE);
        invocation2.setParameterTypes(GENERIC_PARAMETER_TYPES);
        invocation2.setArguments(new Object[]{methodName, types, args});
        Result result = invoker.invoke(invocation2);
... 
}

dubbo对参数进行了序列化，继续看下工具类序列化代码，序列化其实是将Java类序列化为一个map对象，我们只需要查看primitive类型的部分即可，可以看到对于primitive类型，dubbo序列化工具类什么也没做直接返回原类型数据

private static Object generalize(Object pojo, Map<Object, Object> history) {
    ...
    if (ReflectUtils.isPrimitives(pojo.getClass())) {
        return pojo;
    }
...
}

小结

dubbo表示这锅我不背-_-!!!

泛化调用前参数转换问题？

网关侧参数转换代码

// 参数原JSON字符串来自JsonArray.get(0)：[{\"cityId\":1,\"riderId\":5150213,\"riderType\":0,\"type\":95,\"cost\":0.01,\"tradeNoStr\":\"6899423540852876801618889049000\"}]
private Object convert(JsonElement element) {
    if (element == null || element.isJsonNull()) {
        return null;
    } else if (element.isJsonPrimitive()) {
        JsonPrimitive primitive = (JsonPrimitive) element;
        if (primitive.isNumber()) {
            // commons-lang3-3.8.1.jar
            // org.apache.commons.lang3.math.NumberUtils#createNumber
            return NumberUtils.createNumber(primitive.getAsString());
        } else if (primitive.isBoolean()) {
            return element.getAsBoolean();
        } else {
            return element.getAsString();
        }
    } else if (element.isJsonArray()) {
        return convertArray(element.getAsJsonArray());
    } else {
        JsonObject jsonObject = element.getAsJsonObject();
        Map<String, Object> map = new LinkedHashMap<>();
        jsonObject.entrySet().forEach(entry -> {
            map.put(entry.getKey(), convert(entry.getValue()));
        });
        return map;
    }
}

尝试复现问题

public static void main(String[] args) {
    Gson gson = new GsonBuilder().setDateFormat("yyyy-MM-dd HH:mm:ss,SSS").create();
    JsonArray je = gson.fromJson("[{\"cityId\":1,\"riderId\":5150213,\"riderType\":0,\"type\":95,\"cost\":0.01,\"tradeNoStr\":\"6899423540852876801618889049000\"}]", JsonArray.class);
    System.out.println("333="+je);
    Object[] objs = convertArray(je);
    for (Object data : objs) {
        System.out.println(data.getClass());
        if (data instanceof Map) {
            Map<?, ?> datas = (Map<?, ?>) data;
            datas.forEach((k, v) -> System.out
                .printf("k=%s,v=%s%n", k, v));
        }
    }
}

数据结果

333=[{"cityId":1,"riderId":5150213,"riderType":0,"type":95,"cost":0.01,"tradeNoStr":"6899423540852876801618889049000"}]
class java.util.LinkedHashMap
k=cityId,v=1
k=riderId,v=5150213
k=riderType,v=0
k=type,v=95
k=cost,v=0.01
k=tradeNoStr,v=6899423540852876801618889049000

？？？cost值没有问题，为毛到了服务端出现了问题？服务端有内鬼？查看服务端泛化调用责任链逻辑：com.alibaba.dubbo.rpc.filter.GenericFilter

public Result invoke(Invoker<?> invoker, Invocation inv) throws RpcException {
    if (inv.getMethodName().equals(Constants.$INVOKE)
            && inv.getArguments() != null
            && inv.getArguments().length == 3
            && !ProtocolUtils.isGeneric(invoker.getUrl().getParameter(Constants.GENERIC_KEY))) {
        String name = ((String) inv.getArguments()[0]).trim();
        String[] types = (String[]) inv.getArguments()[1];
        Object[] args = (Object[]) inv.getArguments()[2];
        try {
            Method method = ReflectUtils.findMethodByMethodSignature(invoker.getInterface(), name, types);
            Class<?>[] params = method.getParameterTypes();
            if (args == null) {
                args = new Object[params.length];
            }
            String generic = inv.getAttachment(Constants.GENERIC_KEY);
            if (StringUtils.isEmpty(generic)
                    || ProtocolUtils.isDefaultGenericSerialization(generic)) {
                // 反序列化对象
                args = PojoUtils.realize(args, params, method.getGenericParameterTypes());
            } ...
}

查看反序列化逻辑，我们客户端泛化调用时将入参序列化为一个map对象，此时服务端将其反序列化为接口的实际入参类型：com.alibaba.dubbo.common.utils.PojoUtils#realize0，源码可以看到通过反射调用目标方法参数类型的set方法或filed字段写入map的value值

private static Object realize0(Object pojo, Class<?> type, Type genericType, final Map<Object, Object> history) {
    ...
    if (pojo instanceof Map<?, ?> && type != null) {
        ...
        } else {
            Object dest = newInstance(type);
            history.put(pojo, dest);
            for (Map.Entry<Object, Object> entry : map.entrySet()) {
                Object key = entry.getKey();
                if (key instanceof String) {
                    String name = (String) key;
                    Object value = entry.getValue();
                    if (value != null) {
                        Method method = getSetterMethod(dest.getClass(), name, value.getClass());
                        Field field = getField(dest.getClass(), name);
                        if (method != null) {
                            if (!method.isAccessible())
                                method.setAccessible(true);
                            Type ptype = method.getGenericParameterTypes()[0];
                            value = realize0(value, method.getParameterTypes()[0], ptype, history);
                            try {
                                method.invoke(dest, value);
                            } catch (Exception e) {
                                e.printStackTrace();
                                throw new RuntimeException("Failed to set pojo " + dest.getClass().getSimpleName() + " property " + name
                                        + " value " + value + "(" + value.getClass() + "), cause: " + e.getMessage(), e);
                            }
                        } else if (field != null) {
                            value = realize0(value, field.getType(), field.getGenericType(), history);
                            try {
                                field.set(dest, value);
                            } catch (IllegalAccessException e) {
                                throw new RuntimeException("Failed to set filed " + name + " of pojo " + dest.getClass().getName() + " : " + e.getMessage(), e);
                            }
                        }
                    }
                }
            }
            ...
}

这里有一个内部类型转换Object-》BigDecimal，如果Map中的value值不是BigDecimal类型则可能会出现问题，回过头修改下复现代码查看org.apache.commons.lang3.math.NumberUtils#createNumber方法将字符“0.01”转换为Number类型的实际类型

public static void main(String[] args) {
    Gson gson = new GsonBuilder().setDateFormat("yyyy-MM-dd HH:mm:ss,SSS").create();
    JsonArray je = gson.fromJson("[{\"cityId\":1,\"riderId\":5150213,\"riderType\":0,\"type\":95,\"cost\":0.01,\"tradeNoStr\":\"6899423540852876801618889049000\"}]", JsonArray.class);
    System.out.println("333="+je);
    Object[] objs = convertArray(je);
    for (Object data : objs) {
        System.out.println(data.getClass());
        if (data instanceof Map) {
            Map<?, ?> datas = (Map<?, ?>) data;
            datas.forEach((k, v) -> System.out
                .printf("k=%s,v=%s,k.class=%s,v.class=%s%n", k, v, k.getClass(), v.getClass()));
        }
    }
}

输出结果

333=[{"cityId":1,"riderId":5150213,"riderType":0,"type":95,"cost":0.01,"tradeNoStr":"6899423540852876801618889049000"}]
class java.util.LinkedHashMap
k=cityId,v=1,k.class=class java.lang.String,v.class=class java.lang.Integer
k=riderId,v=5150213,k.class=class java.lang.String,v.class=class java.lang.Integer
k=riderType,v=0,k.class=class java.lang.String,v.class=class java.lang.Integer
k=type,v=95,k.class=class java.lang.String,v.class=class java.lang.Integer
k=cost,v=0.01,k.class=class java.lang.String,v.class=class java.lang.Float
k=tradeNoStr,v=6899423540852876801618889049000,k.class=class java.lang.String,v.class=class java.lang.String

小结

问题已经确认正是String类型转Number类型过程中对于小数类型，由于程序无法识别你的小数类型是哪种浮点类型，默认按照最小满足方式转换，0.01转换为Float类型，服务端实际字段类型为BigDecimal类型，Float至BigDecimal类型隐式转换出现了精度问题。验证代码如下

float f = 0.01f;
System.out.println(new BigDecimal(f));
// 输出结果
0.00999999977648258209228515625

总结

问题原因：字符串转浮点型导致，因为程序无法感知你是哪种浮点型，浮点型float转BigDecimal存在精度问题

问题解法：较为粗暴的将所有Number类型转为BigDecimal，Gson当然也早已考虑到这些场景咯，针对primitive类型提供了转换方法=com.google.gson.JsonPrimitive#getAsBigDecimal

 

———————————————— 版权声明：本文为CSDN博主「会灰翔的灰机」的原创文章，遵循CC 4.0 BY-SA版权协议，转载请附上原文出处链接及本声明。 原文链接：https://blog.csdn.net/u010597819/article/details/116095242

标签：dubbo,调用,泛化,Object,null,class,String
来源： https://www.cnblogs.com/Not-If/p/16336870.html

本站声明： 1. iCode9 技术分享网（下文简称本站）提供的所有内容，仅供技术学习、探讨和分享；
2. 关于本站的所有留言、评论、转载及引用，纯属内容发起人的个人观点，与本站观点和立场无关；
3. 关于本站的所有言论和文字，纯属内容发起人的个人观点，与本站观点和立场无关；
4. 本站文章均是网友提供，不完全保证技术分享内容的完整性、准确性、时效性、风险性和版权归属；如您发现该文章侵犯了您的权益，可联系我们第一时间进行删除；
5. 本站为非盈利性的个人网站，所有内容不会用来进行牟利，也不会利用任何形式的广告来间接获益，纯粹是为了广大技术爱好者提供技术内容和技术思想的分享性交流网站。

关于我们 | 联系我们 | 留言反馈

专注分享技术，共同学习，共同进步。侵权联系[81616952@qq.com]

Copyright (C)ICode9.com, All Rights Reserved.

ICode9版权所有