feat: min/max消除, top n #249

cyz-2023 · 2024-11-20T01:17:23Z

当select字段只包括min或max函数时，转换为order by xx limit 1。
sort算子存在limit时，使用topn
agg/distinct没有聚合函数时，支持limit
db支持动态修改user quota。
无结果时，agg返回0值
json类型和bit类型支持
like前缀匹配转range
支持distinct count(*)
float & double返回值精度同to_string函数

lgqss · 2024-11-20T07:29:17Z

src/common/expr_value.cpp

-                body_len = snprintf(tmp_buf, sizeof(tmp_buf), "%.12g", _u.double_val);
+            std::ostringstream oss;
+            if (float_precision_len != -1) {
+                oss << std::fixed << std::setprecision(float_precision_len);


精度改回12，否则最大值四舍五入会溢出

lgqss · 2024-11-27T09:49:26Z

src/logical_plan/select_planner.cpp

+    if (select_expr.nodes_size() != 2) {
+        return 0;
+    }
+    if (select_expr.nodes(0).node_type() != pb::AGG_EXPR) {


上面有了_select_exprs[0].nodes(0).node_type() != pb::AGG_EXPR

src/logical_plan/select_planner.cpp

src/exec/sort_node.cpp

src/runtime/topn_sorter.cpp

lgqss · 2024-11-27T11:59:07Z

include/sqlparser/sql_parse.y

@@ -4028,7 +4028,7 @@ FloatingPointType:
 BitValueType:
    BIT
    {
-        $$ = MYSQL_TYPE_BIT;
+        $$ = MYSQL_TYPE_LONG;


为啥转MYSQL_TYPE_LONG

没有针对MYSQL_TYPE_BIT的逻辑，，为了支持这个类型，就用long去存储了

sql parser可以不变，可以ddl那里映射到long

已修改，ddl_planner中将bit映射为pb::int32

搞错了，修改为了int64

lgqss · 2024-11-27T12:04:30Z

src/expr/internal_functions.cpp

@@ -848,6 +849,158 @@ ExprValue json_extract(const std::vector<ExprValue>& input) {
    return tmp;
 }

+ExprValue json_extract1(const std::vector<ExprValue>& input) {


为啥不能调用json_extract

json对象的 ->和->>操作符返回的结果不一样，就写了两个函数

我看了下mysql函数，原来的json_extract返回不兼容是吧，mysql带引号

是的，就是带引号和不带引号两种都有，那个json_extract不带引号

lgqss · 2024-11-27T12:20:09Z

src/exec/agg_node.cpp

@@ -168,7 +168,7 @@ int AggNode::open(RuntimeState* state) {
        ExecNode* packet = get_parent_node(pb::PACKET_NODE);
        // baikaldb才有packet_node;只在baikaldb上产生数据
        // TODB:join和子查询后续如果要完全推到store运行得注意
-        if (packet != nullptr) {
+        if (packet != nullptr || _is_merger) {


select id from (select max(id) as id from test where id < 1) t;

子查询里面如果没有符合条件的数据，不会返回null。导致外面的查询没有结果。实际应该返回个null

lgqss · 2024-11-27T12:27:39Z

src/exec/fetcher_store.cpp

+            write_binlog_param.username = client_conn->user_info->username;
+            write_binlog_param.ip = client_conn->ip;
+            write_binlog_param.client_conn = client_conn;
+            write_binlog_param.fetcher_store = this;


啥情况会core来着

optype = rollback的时候，会走到write_binlog里面，取param->client_conn时出core

lgqss · 2024-11-27T12:31:04Z

src/protocol/network_server.cpp

@@ -832,6 +832,7 @@ static void on_health_check_done(pb::StoreRes* response, brpc::Controller* cntl,
    std::unique_ptr<pb::StoreRes> response_guard(response);
    std::unique_ptr<brpc::Controller> cntl_guard(cntl);
    pb::Status new_status = pb::NORMAL;
+    old_status = SchemaFactory::get_instance()->get_instance_status(addr).status;


会传过来，为啥再拿一遍

传过来的是旧的，有个场景， store A在一轮检测的最后一个实例检测， 1s超时会设置为faulty，但在设置为faulty之前。已经开始下一轮检测了，第二轮开始的时候取的status，中间不会变更status，第二轮检测完后应该设置为dead，但因为是第二轮检测刚开始的时候拿的status，所以old_status还等于normal。

lgqss · 2024-11-29T11:44:53Z

src/expr/expr_node.cpp

+        ScalarFnCall *ltexpr = new ScalarFnCall();
+        SlotRef* ltslot = slot->clone();
+        ExprValue end_val = prefix_value;
+        int i = end_val.str_val.length() - 1;


那这个不对，c==255的情况，应该是c=0，然后前一个char+1

lgqss · 2024-11-29T11:50:02Z

src/expr/internal_functions.cpp

@@ -848,6 +849,158 @@ ExprValue json_extract(const std::vector<ExprValue>& input) {
    return tmp;
 }

+ExprValue json_extract1(const std::vector<ExprValue>& input) {


我看了下mysql函数，原来的json_extract返回不兼容是吧，mysql带引号

lgqss · 2024-12-02T06:59:44Z

src/logical_plan/ddl_planner.cpp

@@ -1899,6 +1899,12 @@ pb::PrimitiveType DDLPlanner::to_baikal_type(parser::FieldType* field_type) {
    case parser::MYSQL_TYPE_TDIGEST: {
        return pb::TDIGEST;
    } break;
+    case parser::MYSQL_TYPE_BIT: {
+        return pb::INT32;


不是INT64吗

已修改，是应该int64， mysql最大支持到64位

lgqss requested changes Nov 27, 2024

View reviewed changes

cyz-2023 added 3 commits November 28, 2024 11:26

feat: min/max消除, top n

694b8c0

feat: support distinct count(*)

a160c92

feat: like to range

da28f18

cyz-2023 force-pushed the master branch from feb37cf to 2b44d3e Compare December 2, 2024 03:40

lgqss requested changes Dec 2, 2024

View reviewed changes

cyz-2023 added 10 commits December 2, 2024 15:13

feat: bit type transfer to int

b8b15e4

feat: add json type

0ffeb66

fix: float & double返回值精度同to_string函数

884bdb4

fix: agg return 0 when result empty

57697e7

feat: 可动态修改user quota

3d0d2c5

feat: agg没有聚合函数时支持limit

47c3a2b

fix: rollbak send binlog coredump

0f28e08

fix: prepare 不支持full_export

55362a4

feat: store health check get old status when update

1430a7d

fix: cstore + ttl + separate乱码问题

24c2c1f

cyz-2023 force-pushed the master branch from 2b44d3e to 24c2c1f Compare December 2, 2024 07:28

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: min/max消除, top n #249

feat: min/max消除, top n #249

cyz-2023 commented Nov 20, 2024 •

edited

Loading

lgqss Nov 20, 2024

cyz-2023 Dec 2, 2024

lgqss Nov 27, 2024

cyz-2023 Dec 2, 2024

lgqss Nov 27, 2024

cyz-2023 Nov 28, 2024

lgqss Nov 28, 2024

cyz-2023 Dec 2, 2024

cyz-2023 Dec 2, 2024

lgqss Nov 27, 2024

cyz-2023 Nov 28, 2024

lgqss Nov 29, 2024

cyz-2023 Dec 2, 2024

lgqss Nov 27, 2024

cyz-2023 Nov 28, 2024

lgqss Nov 27, 2024

cyz-2023 Nov 28, 2024

lgqss Nov 27, 2024

cyz-2023 Nov 28, 2024

lgqss Nov 29, 2024

lgqss Nov 29, 2024

lgqss Dec 2, 2024

cyz-2023 Dec 2, 2024

feat: min/max消除, top n #249

Are you sure you want to change the base?

feat: min/max消除, top n #249

Conversation

cyz-2023 commented Nov 20, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

cyz-2023 commented Nov 20, 2024 •

edited

Loading