Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat(function): support md5 #5846

Open
wants to merge 4 commits into
base: master
Choose a base branch
from
Open

feat(function): support md5 #5846

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
84bb690
feat(function): support md5
fansehep Apr 2, 2024
d2b17d4
format code
fansehep Apr 10, 2024
993bea6
clean cmake option
fansehep Apr 11, 2024
e607095
fix cmake
fansehep Apr 20, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions CMakeLists.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,8 @@ option(ENABLE_PACKAGE_TAR "Enable package artifacts to tar." OFF)
option(ENABLE_CREATE_GIT_HOOKS "Enable create git hooks." ON)
option(ENABLE_INCLUDE_WHAT_YOU_USE "Enable include-what-you-use find nouse include files" OFF)

SET(CMAKE_EXPORT_COMPILE_COMMANDS TRUE)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This CMAKE_EXPORT_COMPILE_COMMANDS option could be specified in cmake configuration step and need not to always turn it on. please cleanup it.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

please let the CI run.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@fansehep hi, according to yixinglu's response, you can simply remove the added CMAKE_EXPORT_COMPILE_COMMANDS configuration statement. Currently, it is set to true, but as per yixinglu's suggestion, it is already configurable elsewhere, so there is no need to hardcode it here.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks for your reminders, but from the CI compile output, the mold linker link error :(

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nebula CI sometimes needs to run a few more times to pass. I will rerun it a few more times. Wait for my good news 🥺.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks for your reminders, but from the CI compile output, the mold linker link error :(

Perhaps you should add some link dependencies, as commented in this issue. #5840

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@fansehep To fix the failure of CI, u'd better to check where the target $<TARGET_OBJECTS:function_manager_obj> is used in the whole code base, and add the the ${PROXYGEN_LIBRARIES} dependency for there.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@fansehep CI has been fixed, shall we go on?


add_definitions(-DNEBULA_HOME=${CMAKE_SOURCE_DIR})

if(ENABLE_STANDALONE_VERSION)
Expand Down
23 changes: 22 additions & 1 deletion src/common/function/FunctionManager.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@

#include <float.h>
#include <folly/json.h>
#include <proxygen/lib/utils/CryptUtil.h>

#include <boost/algorithm/string.hpp>
#include <boost/algorithm/string/replace.hpp>
Expand Down Expand Up @@ -442,9 +443,9 @@ std::unordered_map<std::string, std::vector<TypeSignature>> FunctionManager::typ
{TypeSignature({Value::Type::STRING}, Value::Type::MAP),
TypeSignature({Value::Type::STRING}, Value::Type::NULLVALUE)}},
{"score", {TypeSignature({}, Value::Type::__EMPTY__)}},
{"md5", {TypeSignature({Value::Type::STRING}, Value::Type::STRING)}},
};

// static
StatusOr<Value::Type> FunctionManager::getReturnType(const std::string &funcName,
const std::vector<Value::Type> &argsType) {
auto func = funcName;
Expand Down Expand Up @@ -3000,6 +3001,26 @@ FunctionManager::FunctionManager() {
return Value::kNullValue;
};
}
{
auto &attr = functions_["md5"];
attr.minArity_ = 1;
attr.maxArity_ = 1;
attr.isAlwaysPure_ = true;
attr.body_ = [](const auto &args) -> Value {
switch (args[0].get().type()) {
case Value::Type::NULLVALUE: {
return Value::kNullValue;
}
case Value::Type::STRING: {
std::string value(args[0].get().getStr());
return proxygen::md5Encode(folly::StringPiece(value));
}
default: {
return Value::kNullBadType;
}
}
};
}
} // NOLINT

// static
Expand Down
3 changes: 3 additions & 0 deletions src/common/function/test/CMakeLists.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@ nebula_add_test(
LIBRARIES
gtest
gtest_main
${PROXYGEN_LIBRARIES}
)

nebula_add_test(
Expand All @@ -37,6 +38,7 @@ nebula_add_test(
$<TARGET_OBJECTS:fs_obj>
LIBRARIES
gtest
${PROXYGEN_LIBRARIES}
)

nebula_add_test(
Expand All @@ -52,5 +54,6 @@ nebula_add_test(
LIBRARIES
gtest
gtest_main
${PROXYGEN_LIBRARIES}
)

8 changes: 6 additions & 2 deletions src/common/function/test/FunctionManagerTest.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -170,8 +170,8 @@ std::unordered_map<std::string, std::vector<Value>> FunctionManagerTest::args_ =
{"json_extract1", {"{\"a\": 1, \"b\": 0.2, \"c\": {\"d\": true}}"}},
{"json_extract2", {"_"}},
{"json_extract3", {"{a: 1, \"b\": 0.2}"}},
{"json_extract4", {"{\"a\": \"foo\", \"b\": 0.2, \"c\": {\"d\": {\"e\": 0.1}}}"}}};

{"json_extract4", {"{\"a\": \"foo\", \"b\": 0.2, \"c\": {\"d\": {\"e\": 0.1}}}"}},
{"md5", {"abcdefghijkl"}}};
#define TEST_FUNCTION(expr, ...) \
do { \
EXPECT_TRUE(testFunction(#expr, __VA_ARGS__)); \
Expand Down Expand Up @@ -248,6 +248,7 @@ TEST_F(FunctionManagerTest, testNull) {
TEST_FUNCTION(concat, args_["nullvalue"], Value::kNullValue);
TEST_FUNCTION(concat_ws, std::vector<Value>({Value::kNullValue, 1, 2}), Value::kNullValue);
TEST_FUNCTION(concat_ws, std::vector<Value>({1, 1, 2}), Value::kNullValue);
TEST_FUNCTION(md5, args_["nullvalue"], Value::kNullValue);
}

TEST_F(FunctionManagerTest, functionCall) {
Expand Down Expand Up @@ -474,6 +475,9 @@ TEST_F(FunctionManagerTest, functionCall) {
args_["json_extract4"],
Value(Map({{"a", Value("foo")}, {"b", Value(0.2)}, {"c", Value(Map())}})));
}
{
TEST_FUNCTION(md5, args_["md5"], "9fc9d606912030dca86582ed62595cf7");
}
{
auto result = FunctionManager::get("hash", 1);
ASSERT_TRUE(result.ok());
Expand Down
Loading